"Typical Chat-GPT query uses about 10 times more energy than Google search That’s just for a basic generative AI function. More advanced queries require substantially more power that have to go through an AI Cluster Farm to process large-scale computing between multiple machines."
"AI is typically deployed in 20-30 cabinet clusters at or above 40 KW per cabinet. This represents a fourfold increase in KW/ cabinet with the deployment of AI. The difference is staggering."
@gerrymcgovern I do agree that chatbot queries do genuinely require more energy, but I think there's a little more consideration here.
If you're measuring energy consumption, you need to do a kind of "lifecycle analysis" -- if the choice is between using a traditional search engine and asking a chatbot, you should compare the entire workflow with each.
If I do a regular web search for something, I will frequently click three to four of the results and open them in new tabs, because I'm not sure exactly which one will answer my question; I might do another search. Each of those loads a website, with all the accompanying HTML, JS, and so on.
With chatbots, I find it's more common for the response to have exactly what I want. "One and done", as they say.
For example, the other day I wanted to know what command in vim will copy text to the system clipboard. (It's "+y when in visual mode, BTW.)
From that perspective, it's not super obvious that the total energy consumption for using a chatbot is that much more than usual searching.
(I hasten to add that it *is* obvious that the overall energy consumption of AI-related stuff -- training the LLMs, operating the chatbots, and so on -- is absolutely staggering and an enormous environmental concern!)
@gerrymcgovern now I'm veering even farther away, but here's another thing to think about when considering the energy usage of generative AI.
Think about what the "deep research" -- such as Google Gemini's (https://gemini.google/overview/deep-research/) can do, for example with medical advice.
Okay, so medical advice is a bit fraught, but think of a person putting in some symptoms, what they know about their health, and so on, and getting a pretty detailed report. That's a *lot* of datacenter energy use.
But we need to compare it with the alternative, which in the US, likely involves driving a car to some clinic, which very well could be miles away.
Think about driving a car, say, 20 miles, along with the implicit energy use of the medical clinic. And your doctor -- again, in the US -- may not be much more helpful than that report.
These two things -- the AI-generated research, and your doctor's advice and comments -- are certainly not always comparable. But if they are? I suspect the AI uses less energy (and it could be cleaner, since the electricity can be generated from renewables or efficient natural gas plants).
I'll admit that this example is real. I actually did this. I have some mysterious medical conditions and symptoms, and what I got from the chatbot's research was, when I compare it to what I've been getting from my doctors, fairly comparable. And I didn't have to make an appointment, to wait, to spend the time driving, to pay for the appointment -- and I got a report with citations and ideas.
Just some more thoughts if we're thinking about the energy usage of AI systems...
@gerrymcgovern I don't disagree! You have some good observations. But let me, in a friendly way, offer these ideas:
1. Take what you wrote, and replace "AI" with "television". I could imagine someone at the dawn of the tv era saying the exact same thing!
2. I think you could say we have *always* been a devouring civilizaton.
Hell, we humans have been devouring since before civilization! Look at the fossil record of how the extinction of megafauna like, well, mastodons, closely tracks with the expansion of early human populations around the world. We've been devouring resources -- whether ecological ones, or mineral ones, or whatever -- for as long as we've been a species.
@gerrymcgovern
Thanks for sharing this. I appreciate the level of detail provided here, from folks working in the field.
One of the most interesting things here to me is the suggestion that we already have one strong model for drastically reducing power and water requirements. The question is, how long will it take for it (or another approach) to be taken up by the industry.
@gerrymcgovern
AI Chatbots are not the problem, if we're talking about energy use/effects on the climate.
10x a Google search is still an extremely small amount (as in 3mins of light bulb small!), and the 10x figure is also quite outdated (we're approaching 1x recently it seems)
The real AI-electricity offenders are apparently ad/recommender systems etc. and industrial applications.
I can recommend this post for a detailed discussion:
https://andymasley.substack.com/p/a-cheat-sheet-for-conversations-about
AI & crypto data centers already use so much electricity, they cause brownouts & raise utility rates for locals.
A frying planet does not need anything that drives increased fossil fuel usage.
https://www.wpr.org/news/power-wisconsin-data-centers-gas-plants-pollution-higher-costs
https://www.semafor.com/article/12/18/2024/ai-energy-demand-raises-risk-of-blackouts-across-the-us
https://www.bloomberg.com/graphics/2024-ai-power-home-appliances/
We Energies is building new gas-fired power plants to meet growing energy demand that largely stems from data centers. As developers pursue data centers in Wisconsin, that’s raising concerns about environmental impacts tied to their operations.
Right there with ya.
"ChatGPT query 10x as much as a Google search." is roughly 3 Wh per prompt and equivalent to
- leaving 11W led light light on for 16 minutes,
- streaming video in MacBook Air for about 30 minutes.
- 600W microwave for 20 seconds.
- 230W refrigerator running 46 seconds.
- driving 190 Wh/km EV vehicle 330 meters.
The cost of 100 queries per day is similar to leaving a lights on in a room for a day.
Not just interesting, but essential.
It's important to put thing into proportion so that we can make wise policy, business and consumer choices. Depending on what kind of activity those data centers replace or enable, they can save or waste energy.
For example, does a help center with AI consume more or less energy per call than a humans sitting in air conditioned rooms in the front of computer? How much?
Datacenter using 1 GWh of energy is equal to 333 million individual 3 Wh uses. Just because lots of energy is used in one place does not make it inherently worse.
Instead of "country sized" we should think in per worker, per consumer, per hour of work or play, and see where it fits in energy use.
One could save a lot of money by saying NO to AI.
One could gain a lot of customer goodwill by making this public.
@gerrymcgovern are you thinking of Nuclear Fusion Reactor?
I am talking about small modular nuclear reactors that use existing nuclear fission process. They use safer fuels and cooling methods.
The sales pitch is that they are super cheap and scalable. I believe the only thing in the way is regulation.
barelysecure
Gerry McGovern
normis
Leendaal
Calvin
Richard Lewis
Jonas Köritz
Dan Drake 🦆
Mateusz Polito
Trevor Burrows
nightoo
Cy
Nicole Parsons
lemgandi
Riccardo Mori
tinydoctor
Violet Madder
ma𝕏pool
Klongeiger
The_Moose_Man