ugjkaAI Bubble: Nobody will pay for unsubsidised AI | Ed Zitron
NaibofTabrCory Doctorow made a very specific point about this on This Week in Tech 1074, in the context of comparing the growth of the Internet with the current AI market:
The web lost money for a long time. And it’s true, they did, but they had good unit economics, right? Every user of the web made the web less unprofitable. Every use of the web made the web less unprofitable. And every generation of the web made the web more profitable. Contrast this with AI, where every time they sign up a user, they lose more money. Every time the user uses their account, they lose even more money.
And every generation of AI accelerates the rate at which they are losing money.
I think it sums up how unsustainable this is very nicely.
Why are they losing money for every user? Because it’s free? If so, isn’t that how the web operated for its free websites and at slme point pushed for ads to cover their expenses?
jonathanThe problem is how much computation is required to handle every user request.
When the Internet was starting out, most websites weren’t much more than text, maybe some low-resolution pictures. Even in the '90s, serving that content to users was computationally cheap. A company’s web server could just be a desktop in the basement.
AI models are expensive to train and expensive to operate. Just maintaining the environmental needs for the massive data centers is a significant cost. Charging users for access is not nearly enough to cover the expenses, by orders of magnitude, and they’re already in massive debt.
Forgot about the compute side of things, you’re right
They’re heavily subsidizing the costs to gain users who otherwise probably won’t be interested in the service at a sustainable cost. Every company is hiding their inference costs, but it’s clear that every user is currently burning far more than they’re generating in revenue. The hope is that inference costs will go down, and while that’s a fairly safe bet, there’s two problems:
Even worse, models themselves are becoming commodities. Although users seem to have preferences for one model over others, there’s still not really a good way to benchmark them. Without a clear ability to differentiate models on performance or ability they’re completely interchangeable, which lowers margins. Why pay more to run company X’s latest and greatest, when company Y’s last generation performs almost identically?
The reason the web was able to cover costs with advertising is because the cost to serve a web page was minimal. A bit of networking gear and a couple servers was all you needed to serve a large website. For many sites, you didn’t even need premium hardware, just a cheap, basic PC with an Internet connection. Lots of people ran free hobby websites with minimal cost. Hell, you can run a website on a single board computer like a Raspberry Pi.
By contrast, AI needs huge GPU clusters to respond to a prompt. A four year old H100 GPU will cost around $30,000; typically 8 of those are clustered together in systems that cost more than $300,000. I can’t even find costs for current generation B100 GPUs or B200 clusters, only cloud rentals. Serving an AI model is orders of magnitude more expensive than serving a website.