The environmental comparison I'd be interested in seeing is between a year of heavy personal usage of LLMs (ChatGPT, Claude, Gemini etc) compared to the CO2 emissions from a single passenger flight
Can I do my own personal carbon offsetting by skipping one trip a year?
Does that question even make sense?
@simon I suspect the answer is no unless you're willing to do some SINCERE hand waving in your calculations.
However I'd love to learn that I'm wrong and it would make for a super interesting experiment nonetheless!
@simon @scott I encourage this line of enquiry. We as consumers should know the non-financial costs of our consumption.
This article is a good example where grocery items are priced with environmental impact in mind. Need more of this! https://www.nytimes.com/interactive/2024/09/19/climate/food-costs-protein-environment.html
@simon yes, if it is at all afforded to you by your society, you should avoid doing things that consume energy up until the point where we've balanced our energy use budget with the earths resources. If you do not personally have that ability, it's very understandable: collective energy use requires collective efforts to reduce usage. Change must be pressed for at the societal level.
That said, it is absurd and ridiculous to compare the necessity to heat your home to the vanity of consuming that power in the form of a fancy lying-to-you machine is so ridiculous I can't believe I actually have to say it. I have never used one of these products and I'm still alive and well.
@scott "I have never used one of these products and I'm still alive and well"
Then it's understandable that you wouldn't see why I value them more than an international flight!
I use them for a lot of things, some of it dumb but most of it genuinely useful https://simonwillison.net/series/using-llms/
Not to me. The flight will likely happen anyway.
@simon I think that the way forward here isn't to try and write it off as personal responsibility. Instead, put the focus where it belongs -- on the tech companies building, training & supplying models.
Demand (or better, regulate) that any power used for AI / LLM must come fully from investment by said companies into renewable energy sources, and that they must build out 25% extra capacity to feed back into the grid, beyond what they use.
@awinkler @simon Within the tech industry, AI seems to be the fastest growing segment, and with the way projections look it could move from a fraction to a majority of tech industry energy consumption rather soon. That's why I care specifically about it.
I'm not affiliated with, or well-versed in, any other industries, so I'll leave those to people who know better about it.
For things to improve, we can't wait until we can fix *everything*. It must be sufficient to start with *something*.
@jwarlander @awinkler I believe what Alexander is asking:
Why not have a carbon tax on a higher level and don't get into details about what we do with it.
If in the end AI/LLM gets powered by renewables, fine. If it gets powered by non-renewable but they pay the price, fine too. Because it's a carbon tax it's not even specific to AI/LLMs.
Now it could be a carbon tax or any other reasonable regulation but it seems like capitalism can't figure out a solution for climate change on its own.
@simon while it would be an interesting theoretical question, how many "heavy user of LLMs" would do the same ?
and if "published" wouldn't it just be used as a performance stunt, a potential headline for a "news of the day", hiding the horrible truth (I mean ... small nuclear reactors ? fission startup ? ... Three Mile Island ???!! ) behind the shrub ... (imho)
but you nevertheless can avoid air travel anyway (I do) ... it still works in general to reduce you weight ;p
@furicle @nickalt Exactly, and for two reasons specifically
1) We can just assume all those AI/LLM companies are VC subsidized right now, and
2) The costs on the environment are externalized and not prices in.
It's similar to how beef can be so cheap, because you're not paying for the degraded labd, for the animal abuse, for the bio-diversity loss, for the agro-chemicals destroying our soils, and so on.
@kitten_tech Before you shame air travel specifically I'd check out emissions from car trips.
It's surprising that emissions from cars are not that many orders of magnitudes away from air travel.
Do the calculations for all car trips per year and see how it compares to a flight. The answer might be shocking to you. It was to me.
@furicle Datacenters are using water for cooling, which makes the water consumed for that purpose fungible with other inputs that are capable of fulfilling the same purpose. If @simon is inclined to come up with a CO₂ number that represents "the cost of AI" then including cooling makes that number fully representative.
(Also, it seems possible that evap cooling may become less common as popular awareness of the inequities of water rights in this country appears to have reached critical mass?)
@simon If you are looking at an overall CO2e budget for the way you live, then in principle you could look at where to allocate that 'spend' of CO2e
In economy a flight is approx 0.13kg CO2e per km flown. Double that for Premium economy and triple it for business class. LHR JFK return in economy is 1.6T CO2e per person.
There's estimates on tinterweb that suggest GPT CO2e at about 24tCO2e per day. Training cost est at 522tCO2e
TL;DR travel is always the big CO2e emission for most of us
@LorenAmelang @simon yep, the area for the seat plus the person plus the luggage allowance plus the other bits and bobs. A lay flat biz class seat is 2-3x the space of an econ seat or threabouts.
Actual Co2e varies by plane make and the route flown.
Having done the CO2e calcs for my tech dept, the shock was to see that bluntly, most of our carbon footprint was in the commute to and from work. Overwhelmingly so.
@simon interesting question.
How would you account for the energy spent on training all these models? That costs much more energy than the inference, but it is also spilt on everyone using the models...
@simon FWIW, this is why Google wants nuclear. It's the fossil fuel firms that have taught us to despise electricity, so we reduce stupidly.
Again, electricity != carbon
We could have abundant, cheap, clean electricity everywhere. The issue is intentionally cultural & regulatory in nature rather than technical due to FUD spread by fossil fuel firms for decades that we take as gospel.
This is a great book on how we could be doing electricity without major tech changes: https://www.amazon.com/Electrify-Optimists-Playbook-Energy-Future/dp/0262545047/?tag=mlinar-20
https://piktochart.com/blog/carbon-footprint-of-chatgpt/
According to this, a return flight from SF to Seattle is equivalent to 92593 queries.
I don’t actually vouch for their math, especially since “queries” are such a broad unit which don’t take into account the model, or token count, or the carbon efficiency of the power source. But I think it’s a useful order of magnitude starting point.
@trindflo here's what I got from ChatGPT, problem is I don't trust ANY of the numbers because the underlying information on how much energy an individual prompt uses is effectively unavailable
Throwing in "What are some ethical and philosophical framework I could use to help explore this question?" did bring up some interesting leads though
https://chatgpt.com/share/6712f5dd-9bdc-8006-9a0a-f352ccb5b7f3
@simon I've found the self-reported Llama2 training (!) emissions interesting. See section 2.2.1 here:
https://arxiv.org/abs/2307.09288
That doesn't say much about inference, tho, but it's a start.
You'd also have to include not just GPU energy but e.g. emissions of the data center and supporting software around it, cluster utilization, the emissions of people working on this, traveling, and so on.
There's Scope1/2/3 etc. categories to make this a bit more structured but in the end it's tough.
In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters. Our fine-tuned LLMs, called Llama 2-Chat, are optimized for dialogue use cases. Our models outperform open-source chat models on most benchmarks we tested, and based on our human evaluations for helpfulness and safety, may be a suitable substitute for closed-source models. We provide a detailed description of our approach to fine-tuning and safety improvements of Llama 2-Chat in order to enable the community to build on our work and contribute to the responsible development of LLMs.
@simon If you want to balance out your own emissions (if that even makes sense, see folloing toot), you need to make sure your efforts adher to a concept from the climate space called "additionality":
Your skipped flight only counts if you really would have taken it and you cancel it not because you can but because you have to.
@simon Personal accountability for LLM/AI usage is a tough topic because it moves the accountability away from it's source: the companies behind AI/LLM.
It's exactly the same as nudging people to offset their flights and feeling good about it while not holding the fossil fuel industry accountable.
In the end capitalism doesn't care about the environment and we need a carbon tax or regulation to make sure we have a path forward on this planet. Big tech's climate pledges are non-binding charity.
@graham_knapp I do think about that - am I having an outsized negative effect by showing people how to use this stuff?
I figure people are going to use this stuff anyway, so I hope to have a positive impact by encouraging people to use it responsibly and productively (and not just to generate slop)
Simon Willison
Feoh
Scott 🏴😷
Gasper Zejn
Dalan Mendonca
chebra
Alexander Winkler
SpaceLifeForm
David Colarusso
@frueheneuzeit
Johan Wärlander 🦀
Daniel
Olm-e
furicle
Alaric Snell-Pym
Scott Perry
drs1969 (David Smith) 🇬🇧
LorenAmelang
Ruben
danielfeldroy
shark jacobs
jfroehlich
trindflo
mborus
Trond Olsen
Graham Knapp