Talk - Better than the new Oil

Intro

The last years we’ve seen our climate getting worse and worse. Overall, not a week goes by without a new weather record being broken somewhere around the world. 8 of the 10 hottest years on record happened within the last 13 years (not counting 2023). So you see a pattern here.

Nonetheless, I respect your opinion if you choose to neglect or ignore the pattern, or if climate activism is not for you. That’s okay, I’m merely following science and observations here and as such I will not be perceptive for climate denial. If you are part of the camp of climate denial for whatever reason, this article is probably not interesting for you.

At the same time, it’s also good to caution for some realism. Overreacting to the current status might lead to hasty reactions and ineffective measures.

We are Carbon Illiterate

Now that we have agreed on the terms, let’s start with our own sense of carbon footprint. It appears to be quite difficult to accurately reason about carbon footprint, and sometimes we are completely off. This is not something to be ashamed of, because these things can be quite complex and rapidly changing over time. For example, it might be the case that food produced in local greenhouses result in more emissions compared to the same food in the open air at the other side of the world transported to your door. There are simply too many factors involved. See also this book, or this initiative.

Next to that, there has been ample research asking respondents to estimate the average carbon footprint from an average car burning 1 gallon of fuel, showing that we tend to underestimate our footprint.

As such, there are a couple of numbers to keep in mind:

• The average dutch household uses about 3000 KWh/Year
• The average dutch citizen emits about 8.5 ton of CO2-equivalents (CO2-eq) a year
• Electric vehicles emit about 0.15-0.2 kWh/km
• And gasoline cars emit about 3000 g CO2/l
• Energy production on average in 2021 in the Netherlands was 500 g CO2-eq/KWh, although this number has been going down rapidly ever since.

In conclusion: it is difficult to reason about our footprint.

IT has expanding rapidly over the past decades

At the same time, we see that our lives our more and more dependent on digital services and technology. Investments in IT have been growing rapidly. Whether you are looking at the amount of data we are producing, our networking usage or the number of chips around us, it tends to look something like this:

At the same time, it also has to be noted that many of our physical devices and hardware have become much more efficient over the last decades. Even though Moore’s Law seems to be tapering off, Koomey’s Law (the number of computations per joule of energy dissipated doubled about every 1.57 years) is still going strong and is expected to continue to do so due to more specialistic computer hardware designs coming up.

The Netflix Streaing Emissions Debate

In a 2019 report by the Shift Project, a French thinktank, on the “unsustainable and growing impact” of online video, it was said streaming was responsible for more than 300m tonnes of CO2 (MtCO2) in 2018, or about the yearly emissions of the country of France at the time. The report is taking up by the media, an interview takes place and the statement “30 minutes of Netflix is the same as driving almost four miles” is recorded in the headlines.

Although the original number is not entirely accurate and somewhat taken out of its context, a media-storm is created around Netflix and various estimations and corrections are published. Most notably, the IEA responds with this post and the authors of the Shift project quickly publish their revision as well.

All in all, it’s an interesting debate where both sides are presenting arguments from academic studies. The difference between those studies can mostly be explained by the impact of future improvements in energy efficiency on current estimations. Simply put, if you predict the footprint of IT systems in 2020 with the energy efficiency of 2010, you might need to adjust some of your initial assumptions.

Whether or not streaming Netflix involves a considerable carbon footprint, it should also be noted that all these arguments and estimations are purely based on top-down estimations rather than bottom-up direct measurements. And this is exactly the problem with most of our current initiatives. Older models and estimation methodologies age fast, and we still do not have a whole lot of good tools available for developers/engineers to grasp the emissions of their work. Without actual measurements, it will be difficult to resolve the discussion.

At the same time, there are different actors within the industry voicing how hardware improvements have had a tremendous impact in the past and thus will also save us in the future. I do not want to sound like a luddite here, but personally I do not really share the tech-optimism present within our tech community to a large extent. Namely, history has also shown that technical advancements can have lead to increase usage and in turn cancel the initial improvements. We have seen this most notably in transportation, but in many other fields as well. See also Jevon’s Paradox.

As such: we need to measure our emissions better at the source.

Big Tech Pledges

Can we wait for Big Tech to solve our climate issues? That depends, do we trust big corporates in their public communication and are we as society effective in detecting corporate technological greenwashing? The big cloudproviders were among the first to put out their net-zero or negative-emissions targets for 2030 and are seen proudly presenting their numbers.

Looking more closely, we see however that their claims are based on Power Purchasing Agreements (PPAs), Renewable Energy Offset Certificates and other offsets. PPAs, however, do not imply that the energy used at a facility is green, but merely that it has been compensated by investments into renewable energy sources. In some cases, we see that without these investments those energy sources would not have been created at all, but we also see that companies buying PPAs tend to benefit from lower prices themselves as well. Next to that, most of the offsets and certificates available are not completely watertight. Sure, you’ve planted a forest in Zambia or you protected a forest in Mexico, but there is often no way to check whether that forest will remain in a few years or whether that forest would not have been protected in the first place. In other words, these offsets are generally not effective and often barely audited.

Next to that, cloud providers also present their users with dashboards to visualize their carbon footprint themselves. It’s a great idea to give these numbers to developers and engineers such that they can start to lower their IT emissions, but only if those dashboards present trustworthy numbers. This happens to be quite difficult, especially if part of your dashboard is more focused on comparing how much better you are off with this particular cloud provider compared to fueling your own on-premise datacenter by burning car tires. Yes, cloud is almost always more greener, because we are sharing resources in the background, but we need to be able to trust the numbers of we are to report on them.

The conclusion: maybe it’s not a good idea to wait for big tech to save us, and maybe we should step up and take action ourselves.

A Dream

Some companies are already doings this right. Personally, I love the way Oatly displays the carbon footprint of their product right on the product itself. What if all products inform their consumers in the same way regarding the associated footprint of consuming the product?

The Reality

Measuring energy emisssions from software, programs, services and applications is a relatively new field, although some of the tools have been around for some time. At the same time, we have seen the field reinvent itself a lot as well and new-joiners tend to quickly experience some of its rabbitholes. Let me give you a short summary.

To estimate the CO2-eq emissions of your software, program, service, etc, there are roughly 3 components. Next to that there are also GHG scopes, but let’s not get ahead of ourselves. Those 3 components come down to measuring the eletrical energy used when running certain code, accounting for the computer/data overhead and converting the used eletrical energy into emissions using energy production data.

Although that all sounds quite straightforward, there are a lot of challenges. In my talk I am not going much in depth but I also wrote a paper in case you are interested in finding out more.

Measuring the eletrical energy is generally done in one of 3 ways: either through direct physical meters, through the RAPL estimation models developed by Intel that initially was developed for chip design purposes, or by pure estimation models potentially based on earlier measurements or other proxy variables.

There happen to be quite some tools that allow you to do so nowadays, although they are mostly not very user-friendly. If you are interested in some of these tools, I advise you to have a look at my Sustainable Tech Radar, which provides a summary and ranking of some of the tools available. I intend to update it occasionally.

Other things you can do

Look, writing more sustainable software, code and analyses is not rocket science. Most code that performs better, leads to lower emissions, and probably also lower costs by the way.

For cloud and infrastructure you can think about rightsizing, right-timing and relocation your workloads. Not all cloud provider locations have the same carbon intensity, see here. Next to that, there might also be more local initiatives that are working on datacenters that recover waste-heat and/or work with refurbished hardware. To name a few: LeafCloud, Qarnot, Oxide and Aknostic are in various stages of a solution in this space.

For software you can think of page/code bloat, performance testing, backwards compatibility and low impact upgrade strategies in case your software is running on remote devices. There have also been several studies in the past comparing programming languages, which always tend to show the same thing: the closer to the machine and the less abstractions the better, but it also matters a lot how well your code is given the language your coding in. Indeed, that’s probably more applicable to new projects and depdent on the experience/learning-appetite of your team. Additionally, if your final software product is web-based, there are quite some tools out there to test its performance and sustainability. Probably amongst others: Green Web Foundation, Digital Beacon and Website Carbon.

Finally, for data and logs you should only store what you need. Damn people, please stop just sending over all your logs unless you have an idea what you are going to do with it. You will be paying for it through increased complexity, lower ease of further processing, higher monetary costs and, worst of all, more carbon footprint all over the place in any later processing step. Think before you do, especially if you have a lot of data. The same holds true for creating models. Don’t just run mindless new machine learning iterations, unless you have good reason to believe it will increase the value of your algorithms.

End

That’s it! If you want to know more, have a look at the slides in the link at the top of this page, read my paper or reach out