Mastodawn

Rob Napier Mar 18, 2024

This really drives home something about LLM systems. They’re very expensive to run, both to train and per-query, and hard to make cheaper. I expect them to get more expensive to run. They’re currently sold at a big loss to establish monopoly power and then raise prices dramatically. That’s the *stated* business plan.

If you’re building your business to rely on LLM, you need to factor in what you‘ll do when they pivot to making money, or they pull back because they can’t.
https://geeknews.chat/@theregister/112116266764229145

The Register (@[email protected])

Microsoft promises Copilot will be a 'moneymaker' in the long term Exec tells investors to 'temper' expectations as mission to convince customers of price tag continues Microsoft is asking investors to "temper" expectations for quick financial returns from Copilot amid efforts to convince customers that paying "substantial" sums each month is actually worth it.… #theregister #IT https://go.theregister.com/feed/www.theregister.com/2024/03/18/microsoft_copilot_moneymaker/

Geek News Central Mastodon Chat

Frank Reiff Mar 18, 2024

@cocoaphony @nicklockwood @theregister I find the technology fascinating, but it seems likely that when the hype wears off and the realization that “human-level” performance, especially as it relates to actual understanding, common sense and baseline reliability, is in fact not reachable with current stochastic parrot machine learning, the entire market is going to collapse.

Misuse Case Mar 18, 2024

@frankreiff @cocoaphony @nicklockwood @theregister This is another aspect of it, lots of companies/executives want to believe they can totally replace humans with it, but they can’t and they are going to find out the hard way. That realization and the price jack-up will hit them at the same time.

@MisuseCase @frankreiff @nicklockwood This is very much the story of Uber. They designed their business plan around self-driving cars. As those persistently failed to materialize, the company bled money for years. It's finally profitable after accepting that they're in the business of people-driven taxis, but it'll be a long time to make back what it cost to get there.

"No law can stop us!" has also turned out to be a challenging business model to make sustainable…

https://www.theverge.com/2024/2/8/24065999/uber-earnings-profitable-year-net-income

Uber ends the year in the black for the first time ever

Uber is finally profitable, after 15 years of trying. The company reported its first annual operating profit and positive net income, signaling stronger financial footing.

The Verge

Misuse Case Mar 18, 2024

@cocoaphony @frankreiff @nicklockwood I don’t believe it’s meant to be a sustainable business model I think it’s meant to be a cash grab.

With a lot of these “AI” things it’s much more obvious. Sam Altman and the like are trying to grab as much money and content as possible before a critical mass of people are wise to the con and/or regulation comes down.

Rob Napier Mar 18, 2024

@MisuseCase @frankreiff @nicklockwood I'll buy that for Altman and others, but Microsoft is a company devoted to paying their dividend to investors. There's no "exit" with a pile of cash for them. They're the mark left holding the bag at the end.

So they either need a sustainable business or they have to get out of it. I expect a long grind to figure out how to do either. And I'd be nervous building a business that relies on how they decide to play it.

Misuse Case Mar 18, 2024

@cocoaphony @frankreiff @nicklockwood I guess that’s true for Microsoft, and their users are second- and third-order marks.

RealGene ☣️Mar 18, 2024

@cocoaphony @MisuseCase @frankreiff @nicklockwood
I want an LLM to observe how I use Word, and over time recognize what MY document defaults should look like, and reduce the ribbon to show just the things I actually use.

Misuse Case Mar 18, 2024

@RealGene @cocoaphony @frankreiff @nicklockwood This isn’t actually an LLM as such but it is a machine learning algorithm (of which LLMs are a subset) and that would be a useful thing! I can also think of some other applications in which that type of machine learning algorithm would be useful and there are a few being used that way!

Rob Napier Mar 18, 2024

@MisuseCase @RealGene @frankreiff @nicklockwood I've been doing some rough calculations based on semianalysis's estimate of 0.36 cents/query.

With their $30/month cost (which is probably way too high), if a user makes more than 50 queries/hour, they'll lose money on just the operation costs (not including capex, overhead, R&D, profit, etc.) So their real limit is much lower.

I expect many features people want will require more than that, which makes this very hard.

https://www.semianalysis.com/p/the-inference-cost-of-search-disruption

The Inference Cost Of Search Disruption – Large Language Model Cost Analysis

$30B Of Google Profit Evaporating Overnight, Performance Improvement With H100 TPUv4 TPUv5

SemiAnalysis

Frank Reiff Mar 18, 2024

@cocoaphony @MisuseCase @RealGene @nicklockwood At the moment the influence on share price far outweighs the actual cost.

In future, when cost becomes relevant, there are lots of ways to cut them. After all there is nothing in the contracts that says WHICH model is run. There are efforts underway to “prune” models to make them both more efficient and improve the output. Our own brains work like that, eg glial cells aka white matter.

Frank Reiff Mar 18, 2024

@cocoaphony @MisuseCase @RealGene @nicklockwood

The real problem, I speculate, is going to be falling demand for services that don’t deliver the value that they promise.

IBM’s Watson had this type of hype after its jeopardy win and resulted in a lot of contracts being cancelled when they failed to deliver, for instance on oncology diagnostics. Turns out the real world is disappointingly complex. 🤷‍♂️

Simon Tournier Mar 18, 2024

@cocoaphony From International Energy Agency report:

« When comparing the average electricity demand of a typical Google search (0.3 Wh of electricity) to OpenAI’s ChatGPT (2.9 Wh per request), » p.34 https://www.iea.org/reports/electricity-2024

The electricity seems a good candidate for a zero-order approx. about the "cost" of AI. The money-cost means nothing compared physical resources.

[1/2]
@nicklockwood @frankreiff @RealGene @MisuseCase

Electricity 2024 – Analysis - IEA

Electricity 2024 - Analysis and key findings. A report by the International Energy Agency.

IEA

Simon Tournier Mar 18, 2024

@cocoaphony Climate changes will have strong impacts.

For instance, the lack of water will imply high constraints on electricity production; see International Energy Agency https://www.iea.org/reports/electricity-2024. Hydroelectricity, obviously. And more. Even nuclear; with low water level, poor perfs because low cooling.

And dry area will consume more energy or resources to secure water's supply; e.g. desalination or else.

[2/2]

Electricity 2024 – Analysis - IEA

Electricity 2024 - Analysis and key findings. A report by the International Energy Agency.

IEA

Rob Napier Mar 18, 2024

@zimoun Rising energy costs are a big part of my assumption that they will get more expensive rather than cheaper. There are other limits that I think may kick in more abruptly, though. For example, hardware and data center capacity. As we've learned over the last few years, a disruption in chip supply can impact the whole industry. Fabs are hard to build in response to demand shocks. And whenever cryptocurrency prices spike, AI has to compete for everything with miners. Lots of business risks.

Simon Tournier Mar 18, 2024

@cocoaphony My point is to raise that money-value as "cheap" or "expensive" is an inadequate frame under climate change constraints. A techno. can still be money-profitable because "sponsored" but not physically-profitable.

LLM burn a lot of energy and as zero-order approximation energy = CO2 so LLM cannot be profitable; even now.

I hope that your prediction about money-profit of LLM will happen…

PatNaik Mar 19, 2024

@MisuseCase @RealGene @cocoaphony @frankreiff @nicklockwood What kind of model would you use for that?

Misuse Case Mar 19, 2024

@PratNaik @RealGene @cocoaphony @frankreiff @nicklockwood Some kind of very strictly supervised learning model. I don’t think any LLM falls into that category.

Philippa Cowderoy Mar 19, 2024

@MisuseCase @cocoaphony @frankreiff @nicklockwood The "grab" was meant to be owning the market ahead of when self-driving materialised - and failing that, deregulating it through BS. We'll have to see how the latter goes, because there's a genuine risk of them making the entire taxi market unsustainable for a decade or two across large regions before things recover.

Misuse Case Mar 19, 2024

@flippac @cocoaphony @frankreiff @nicklockwood Yeah, now you have “self-driving” Teslas and these Waymo and Cruise taxis that have problems and exhibit bizarre behaviors that human drivers never would, but get permits nonetheless.

Philippa Cowderoy Mar 19, 2024

@MisuseCase @cocoaphony @frankreiff @nicklockwood We're not going to see Uber running those any time soon: if the car's not responsible for the crashes, the company is and they're big enough to lose their shirt over it

the gripe ibis Mar 19, 2024

@cocoaphony @MisuseCase @frankreiff @nicklockwood haha i immediately thought about uber and the quiet way they distanced themselves from self-driving research.