Mastodawn

One of the problems with those large generative language models is that they need to be kept up-to-date. People will expect the chat bot to know who this week's PM is, or the current hit song. Which means that training will become an ongoing process. So the carbon footprint will become many times larger than it already is. It will also be very expensive, but I don't think OpenAI has much of a choice as customers naturally want the bot to be up-to-date.

Wim🧮Jan 5, 2023

Furthermore, the use of the model also has a considerable footprint. An estimate is given here:
https://medium.com/@chrispointon/the-carbon-footprint-of-chatgpt-e1bc14e4cc2a

3.82 tCO₂e per day assuming a million users with 10 queries per day. It means that after about half a year, the user footprint exceeds the training footprint.
Of course 1M users with 10 queries is a very small number, and likely to grow steeply.

#ClimateEmergency

The carbon footprint of ChatGPT - Chris Pointon - Medium

UPDATE, March 3 2023: The spiralling use of ChatGPT means it is most likely hosted in a range of locations with different electricity carbon intensities. This makes it impossible to give a reasonable…

Medium

Wim🧮Jan 5, 2023

Just to be clear, in the big picture (total global ICT emissions of 5 GtCO₂e) this kind of footprint is still very small. But with a billion users (in practice, this will be other software, not people) with 100 queries per day (a very low estimate I think), we're at 15MtCO₂e, and if there would be 10 such models for different purposes, we can see that it starts to add up quickly.

#ClimateEmergency
#LowCarbonComputing

Wim🧮Jan 5, 2023

Finally, I do not agree with the argument in the article I linked about moving the work to a location with lower carbon intensity. Essentially, the only carbon intensity that matters is the global one. As long as our electricity generation is not 100% carbon-free globally, then using renewables is simply offloading your emissions on someone else.

Wim🧮Feb 19, 2023

If that 15MtCO₂e sounds improbably, consider this: currently, Bing and Google each process 10 billion queries per day.

If each of these queries would require a call to an LLM of the same complexity as GPT-3, as seems to be their intention, then that is already 3MtCO₂e purely from Bing and Google searches alone, without any growth or any other applications.

Wim🧮Feb 19, 2023

Just to keep this with the thread, purely training GPT-3 generates 550 ton of CO2. So compared to the emissions resulting from the queries, that is quite small; but as argued above, it will have to be done almost continuously to keep them up-to-date.

Wim🧮Feb 20, 2023

For more context, emissions from non-ML-enhanced are 0.2g CO₂e per query, so for Bing+Google that is 4ktCO₂e. In other words, purely because of the extra compute required, this would be an increase in emissions of 750x per query.

418 I'm a Teapot Feb 19, 2023

@wim_v12e @csepp

not to forget that developing these things means training them many
times. train, rejigg the network, train, fix a bug, train, many
times. this is not usually reported. most likely it is a couple of
orders of magnitude more than 0.5kT to produce a static trained model.

mostly they don't seem to be continuously feeding these things. not
yet anyways.

Wim🧮Feb 20, 2023

Quite so, that was the start of my thread ^_^

it's B! Cavello 🐝Feb 20, 2023

@wim_v12e I imagine there will be a heavy emphasis on decreasing the compute costs, and we can hope the the sources will be less carbon intensive, but it definitely sounds like a lot @_@

Wim🧮Feb 20, 2023

@b_cavello It's not clear how that would happen. GPUs are already very efficient at this job. Replacing them e.g. with wafer-scale engines would in principle mean efficiency gains on computation but not on I/O, and also the embodied carbon of such a change would be huge.

Wim🧮Feb 20, 2023

As for the As for the "less carbon intensive sources": renewables are only 20% of total US electricity generation. So if Microsoft or OpenAI or Google buys "100% renewables" they simply offload their emissions onto others who can't use renewables. So we should use the average CO2 intensity for the US, or maybe even for the whole world.

Buying electricity from renewables does not reduce global emissions.

it's B! Cavello 🐝Feb 20, 2023

@wim_v12e Agree re: displacement by purchasing renewables, I’m hopeful that more renewables (and less emitting power sources) will be built!

Wim🧮Feb 20, 2023

@b_cavello The most recent optimistic projections say that by 2040 we'll be at 70% renewables. Even if that results in a proportional decrease in emissions from fossil fuels (and currently it doesn't), then that is too late to stop catastrophic warming. Reducing energy consumption is the only way.

it's B! Cavello 🐝Feb 20, 2023

@wim_v12e Re: energy consumption, it’s true that GPUs are really effective, but in this case the costs as well as the externalities are high. There’s a strong incentive to reduce costs for the companies so there is already a lot of emphasis on architecture changes and efficiencies. I don’t think that would be driven by hardware for the costs already mentioned.
Not claiming it’ll “fix everything” but should be incorporated into projections

Wim🧮Feb 20, 2023

@b_cavello Maybe most companies will judge that the cost is too high and go for alternatives for LLMs. And maybe Google and Microsoft will realise that their GPT-augmented search is folly. The ideal outcome would be that the whole bubble would burst.

Michael Veale Feb 20, 2023

@wim_v12e the cost of hardware to provide queries around the world comes with a gigantic cost that few add to these calculations

Wim🧮Feb 20, 2023

@mikarv It's a big factor.
From my analysis, in data centres the emissions from usage still dominate, to such an extent that it is better to replace HW after about 6 years because of the efficiency gains. That will likely still be the case for a few more generations.