Rob NapierMar 18, 2024

This really drives home something about LLM systems. They’re very expensive to run, both to train and per-query, and hard to make cheaper. I expect them to get more expensive to run. They’re currently sold at a big loss to establish monopoly power and then raise prices dramatically. That’s the *stated* business plan.

If you’re building your business to rely on LLM, you need to factor in what you‘ll do when they pivot to making money, or they pull back because they can’t.
https://geeknews.chat/@theregister/112116266764229145

The Register (@[email protected])

Microsoft promises Copilot will be a 'moneymaker' in the long term Exec tells investors to 'temper' expectations as mission to convince customers of price tag continues Microsoft is asking investors to "temper" expectations for quick financial returns from Copilot amid efforts to convince customers that paying "substantial" sums each month is actually worth it.… #theregister #IT https://go.theregister.com/feed/www.theregister.com/2024/03/18/microsoft_copilot_moneymaker/

Geek News Central Mastodon Chat
Show threadFrank ReiffMar 18, 2024
@cocoaphony @nicklockwood @theregister I find the technology fascinating, but it seems likely that when the hype wears off and the realization that “human-level” performance, especially as it relates to actual understanding, common sense and baseline reliability, is in fact not reachable with current stochastic parrot machine learning, the entire market is going to collapse.
Show threadMisuse CaseMar 18, 2024
@frankreiff @cocoaphony @nicklockwood @theregister This is another aspect of it, lots of companies/executives want to believe they can totally replace humans with it, but they can’t and they are going to find out the hard way. That realization and the price jack-up will hit them at the same time.
Show threadRob NapierMar 18, 2024

@MisuseCase @frankreiff @nicklockwood This is very much the story of Uber. They designed their business plan around self-driving cars. As those persistently failed to materialize, the company bled money for years. It's finally profitable after accepting that they're in the business of people-driven taxis, but it'll be a long time to make back what it cost to get there.

"No law can stop us!" has also turned out to be a challenging business model to make sustainable…

https://www.theverge.com/2024/2/8/24065999/uber-earnings-profitable-year-net-income

Uber ends the year in the black for the first time ever

Uber is finally profitable, after 15 years of trying. The company reported its first annual operating profit and positive net income, signaling stronger financial footing.

The Verge
Show threadMisuse CaseMar 18, 2024

@cocoaphony @frankreiff @nicklockwood I don’t believe it’s meant to be a sustainable business model I think it’s meant to be a cash grab.

With a lot of these “AI” things it’s much more obvious. Sam Altman and the like are trying to grab as much money and content as possible before a critical mass of people are wise to the con and/or regulation comes down.

Show threadRob NapierMar 18, 2024

@MisuseCase @frankreiff @nicklockwood I'll buy that for Altman and others, but Microsoft is a company devoted to paying their dividend to investors. There's no "exit" with a pile of cash for them. They're the mark left holding the bag at the end.

So they either need a sustainable business or they have to get out of it. I expect a long grind to figure out how to do either. And I'd be nervous building a business that relies on how they decide to play it.

Show threadRealGene ☣️Mar 18, 2024
@cocoaphony @MisuseCase @frankreiff @nicklockwood
I want an LLM to observe how I use Word, and over time recognize what MY document defaults should look like, and reduce the ribbon to show just the things I actually use.
Show threadMisuse CaseMar 18, 2024
@RealGene @cocoaphony @frankreiff @nicklockwood This isn’t actually an LLM as such but it is a machine learning algorithm (of which LLMs are a subset) and that would be a useful thing! I can also think of some other applications in which that type of machine learning algorithm would be useful and there are a few being used that way!
Show threadRob NapierMar 18, 2024

@MisuseCase @RealGene @frankreiff @nicklockwood I've been doing some rough calculations based on semianalysis's estimate of 0.36 cents/query.

With their $30/month cost (which is probably way too high), if a user makes more than 50 queries/hour, they'll lose money on just the operation costs (not including capex, overhead, R&D, profit, etc.) So their real limit is much lower.

I expect many features people want will require more than that, which makes this very hard.

https://www.semianalysis.com/p/the-inference-cost-of-search-disruption

The Inference Cost Of Search Disruption – Large Language Model Cost Analysis

$30B Of Google Profit Evaporating Overnight, Performance Improvement With H100 TPUv4 TPUv5

SemiAnalysis
Show threadFrank Reiff

@cocoaphony @MisuseCase @RealGene @nicklockwood At the moment the influence on share price far outweighs the actual cost.

In future, when cost becomes relevant, there are lots of ways to cut them. After all there is nothing in the contracts that says WHICH model is run. There are efforts underway to “prune” models to make them both more efficient and improve the output. Our own brains work like that, eg glial cells aka white matter.

Mar 18, 2024 at 4:40pmIvory for iOS