ugjkaAI Bubble: Nobody will pay for unsubsidised AI | Ed Zitron
NaibofTabrCory Doctorow made a very specific point about this on This Week in Tech 1074, in the context of comparing the growth of the Internet with the current AI market:
The web lost money for a long time. And it’s true, they did, but they had good unit economics, right? Every user of the web made the web less unprofitable. Every use of the web made the web less unprofitable. And every generation of the web made the web more profitable. Contrast this with AI, where every time they sign up a user, they lose more money. Every time the user uses their account, they lose even more money.
And every generation of AI accelerates the rate at which they are losing money.
I think it sums up how unsustainable this is very nicely.
paraphrand“But the cost per token keeps going down!” The AI proponents scream.
Why are they wrong?
The cost is not actually going down. The price is going down, because these companies are need new revenue and new customers and new waves of hype to prop up their failing business model. The cost is at best staying level if you want to be very selective and generous, and is inevitably going up. And it’s going to go up much, much faster in the near future.
Sure, Nvidia may put out some newer chips that are technically somewhat more efficient from time to time, but the old cards are still running, they’ve already been purchased and they still have to pay for themselves, and there is no evidence they have (or ever will) and the new ones are even more expensive than that, and they’ll have to pay for themselves too. And at the same time energy is getting more expensive, training costs are getting more expensive, and demand on both sides is increasing which is only going to push those costs even higher.
There is zero possibility any of this makes economic sense. The entire economic output of the world decided to jump on the FOMO bandwagon, that doesn’t mean the wagon is actually going anywhere.
I’m not certain, but it could be that as the cost goes down, they keep bringing out more powerful models, so the cost per query stays constant.
If this is accurate, their need to compete with each other to have “the best” AI is creating a prisoner’s dilemma where everyone loses because the individually-optimal choice is worse than if they all agreed that today’s AI is good enough and concentrated on affordability and minimising energy/water usage instead.
Through no choice of our own
Why are they losing money for every user? Because it’s free? If so, isn’t that how the web operated for its free websites and at slme point pushed for ads to cover their expenses?
jonathanThe problem is how much computation is required to handle every user request.
When the Internet was starting out, most websites weren’t much more than text, maybe some low-resolution pictures. Even in the '90s, serving that content to users was computationally cheap. A company’s web server could just be a desktop in the basement.
AI models are expensive to train and expensive to operate. Just maintaining the environmental needs for the massive data centers is a significant cost. Charging users for access is not nearly enough to cover the expenses, by orders of magnitude, and they’re already in massive debt.
Forgot about the compute side of things, you’re right
They’re heavily subsidizing the costs to gain users who otherwise probably won’t be interested in the service at a sustainable cost. Every company is hiding their inference costs, but it’s clear that every user is currently burning far more than they’re generating in revenue. The hope is that inference costs will go down, and while that’s a fairly safe bet, there’s two problems:
Even worse, models themselves are becoming commodities. Although users seem to have preferences for one model over others, there’s still not really a good way to benchmark them. Without a clear ability to differentiate models on performance or ability they’re completely interchangeable, which lowers margins. Why pay more to run company X’s latest and greatest, when company Y’s last generation performs almost identically?
The reason the web was able to cover costs with advertising is because the cost to serve a web page was minimal. A bit of networking gear and a couple servers was all you needed to serve a large website. For many sites, you didn’t even need premium hardware, just a cheap, basic PC with an Internet connection. Lots of people ran free hobby websites with minimal cost. Hell, you can run a website on a single board computer like a Raspberry Pi.
By contrast, AI needs huge GPU clusters to respond to a prompt. A four year old H100 GPU will cost around $30,000; typically 8 of those are clustered together in systems that cost more than $300,000. I can’t even find costs for current generation B100 GPUs or B200 clusters, only cloud rentals. Serving an AI model is orders of magnitude more expensive than serving a website.
- “The web” is an open protocol aimed for interoperability
- "The AI" is a bunch of competing, mostly closed and private services
That will be interesting to see how this difference impacts the bubble explosion.
- “The web” is an open protocol aimed for interoperability
Yes, although the reality today is that you have to spend a decent amount of money to be on the web effectively. You could run a webpage on an RPi on your home network, but that won’t do for much more than a few visitors a day, and it involves several compromises and some security risks.
There isn’t really an equivalent to the web’s interoperability with these AI systems, except maybe if you’re using it to control smart home devices.
- “The AI” is a bunch of competing, mostly closed and private services
Yeah, although being able to run OpenClaw &etc locally is interesting. Most people won’t ever be able to train their own models, but at least self-hosting some of them is an option.