@niekvdpas To give an even vaguer answer. A fairly limited LLM can be run reasonably well on my four-year-old laptop. A much more capable model can be run on a machine, such as a Mac Studio. This does consume energy, but not very much. Especially given that typical LLM use does not imply sustained full-load operation.
Compared to these examples, optimized hardware or software, scale, better utilization, etc. will be more efficient.
@niekvdpas As far as is known, AI companies do not publish detailed data, so there are more or less plausible guesstimates. Andy Masley (see link) has written several solid articles on the topic. Granted, he is just another person on the internet, but I find his argument reasonably convincing. This is not about specific numbers, but about a rough estimate of the order of magnitude of LLM energy consumption and how that compares to overall personal energy use
https://andymasley.substack.com/p/a-cheat-sheet-for-conversations-about
@caseyliss I’m not getting into that discussion 😃
Display sizes (phones, TVs, monitors) are in inches in Europe too. If you care about cars, 0–60 mph ≈ 0–100 km/h. Fahrenheit? I don’t check the weather over there (but below zero is cold in both systems?).
@atpfm @caseyliss I, for one, appreciate the unit conversions; at the very least, they make me chuckle a bit. So thank you, Casey 😊
BTW: I do have some vague intuition for inches and miles, but all the other units (including Fahrenheit) are just a blank to me.