King of Red LionsJul 12, 2024
Jason Koebler
A blistering Goldman Sachs research report wonders if the massive investment in Generative AI will ever pay off, says stock gains are already baked in "outside of the most bullish AI scenario," and posits that Gen AI is likely a bubble: https://www.404media.co/goldman-sachs-ai-is-overhyped-wildly-expensive-and-unreliable/
Goldman Sachs: AI Is Overhyped, Wildly Expensive, and Unreliable

One of the world's largest investment banks wonders if generative AI will be worth the huge investment and hype: "will this large spend ever pay off?"

404 Media
Jul 12, 2024 at 5:06pmElk
Show threadJason KoeblerJul 12, 2024

This one of Goldman Sachs' top analysts: "This is not a matter of just some tweaks being required here and there; despite its expensive price tag, the technology is nowhere near where it needs to be in order to be useful for even such basic tasks."

Goldman Sachs ran an experiment where they had AI do a task and also did a task manually and found that it cost six times as much to have AI do the same task as humans. Cites "illegible and nonsensical results"

Show threadJason KoeblerJul 12, 2024
The report also questions whether LLMs will ever get substantially better, even with massive amounts of more training data, which might not exist: "it’s not clear where more high-quality data will come from and whether it will be easily and cheaply available to AI models"
Show threadJason KoeblerJul 12, 2024
This comes on the heels of Sequoia stating that there’s a massive, massive spend/revenue discrepancy for AI companies. OpenAI is making a lot of money but other companies are not, cites a $500 million hole: “Outside of ChatGPT, how many AI products are consumers really using today?”
Show threadGiselleJul 13, 2024
@jasonkoebler That's incredibly interesting. Thank you for writing about this.
Show threadInfrapink (he/his/him)Jul 12, 2024

@jasonkoebler

Someone observed that getting an LLM to write a programme simply replaces code-writing with code review.

It looks like the same is true of investing.

Show threadBill Of Earth42Jul 12, 2024
@jasonkoebler What's funny is that AI could probably replace the senior management much more easily than the real workers, but they didn't think of that.
Show threadRalph058 (S/he/it) AF4EZJul 14, 2024
@billOfEarth42 @jasonkoebler AI can replace multiple layers of middle management more easily than it can replace either workers or executives.
For the most part, the most efficient executive does nothing but delegate and AI can be very effective at doing that.
Show threadBill Of Earth42Jul 14, 2024
@Ralph058 @jasonkoebler What I've seen is that the higher you go in management, the more separated you are from the details of the actual business, and with each layer of abstraction, the simpler it gets. The top only worries about two things in a corporation: increasing profit and decreasing risk, ie liability.
Show threadbigiainJul 13, 2024
@jasonkoebler I have cow orkers that can produce illegible and nonsensical results for six times the price of someone competent. I don’t need to pay OpenAI for Incompetence as a Service…
Show threadEric LawtonJul 14, 2024

@bigiain

My new term.
Management bullshit = cow orking.

Typos can be serendipitous.

@jasonkoebler @Urban_Hermit

Show threadbigiainJul 17, 2024
@EricLawton Not a typo. Well, perhaps closer to “an intentional typo”. It was a widely used term back in the mid/late 90s in a Usenet group I hung out in. (Anyone else here an alumni of the Scarey Devil Monastery?) @jasonkoebler @Urban_Hermit
Show threadsmxiJul 12, 2024

@jasonkoebler So Goldman Sachs is saying that when something looks talks walks like a duck it probably is a duck?

Sometimes tech marketing terms are annoying, like Cloud, but they don't actually suggest the duck is not a duck. But with AI (sic) the marketing term is flat out wrong and misleading. As the Curl author noted, the I In LLM stands for intelligence. Also, the U stands for Understanding, and the CS stands for Conceptual Structures.

When training sets shrink and less power needed...

Show threadCurioso 🍉 🇺🇦 (jgg)Jul 12, 2024

@jasonkoebler

Whoever gets the kind of generative AI that would justify all that investement will have the required tools to steal all his partners' shares, with no legal repercussion. And probably he will, since they are exactly the kind of people who will want to keep it for himself.

If AI underperforms, they will lose a ton of money. If it doesn't, they will be really fucked.

Seems like a lose-lose situation.

Show threadSherri W (SyntaxSeed)Jul 12, 2024

@jasonkoebler

"To break even on what they’re spending on AI compute infrastructure, companies need to vastly scale their revenue"

Doesn't revenue have to *also* come from consumers increasing their purchases of a product or service?

I just don't see people or small businesses around me being flush with cash they are ready to spend if only there were #AI features. Quite the opposite in fact.

They might cut costs only to break even with falling income from financially exhausted customers.

Show threadSherri W (SyntaxSeed)Jul 12, 2024
@jasonkoebler At best I think this will be treading water for corporate bottom lines while spreading more misery via the resulting job losses & environmental damage.
Show threadFluffy Kitty CatJul 13, 2024
@syntaxseed @jasonkoebler genAI needs heavy industry on the backend, and there isn't the revenue for that from low stakes usecases, and high stakes usecases need accuracy they can't deliver without more basic research, which takes time, money, and uncertainty.
Show threadJoeMcconJul 13, 2024

@jasonkoebler

My God! I'm astonished! Who could have predicted it?

Show threadLooking for explanations…Jul 13, 2024
@jasonkoebler
And environmentally unsustainable unless they generate their own clean energy.
Show threadDoug 🌈🇨🇦 Jul 13, 2024
@jasonkoebler
POP!!!
Show threadTheriacJul 13, 2024
@jasonkoebler
did Goldman Sachs just try to use Google or something?

Did they not notice the betas they were shown in investor presentations were firmly on rails?

Both yes and no then?
Show threadKofi Loves Efia Jul 13, 2024
@jasonkoebler as long as middle.managers care more about output than quality bet on Ai
Show threadDaniel Howard 🌻Jul 13, 2024

@jasonkoebler "and posits that Gen AI is likely a bubble"

yathink?

Kind obvious for those of us who rode the dot-com rollercoaster.

Show threadcuan_knaggsJul 13, 2024
@jasonkoebler i've been wondering if this is the needle for the "ai" bubble. and then this waffle comes out https://arstechnica.com/?p=2036853
OpenAI reportedly nears breakthrough with “reasoning” AI, reveals progress framework

Five-level AI classification system probably best seen as a marketing exercise.

Ars Technica
Show threadDarwin WoodkaJul 13, 2024
@jasonkoebler Goldman Sachs is also overhyped and wildly expensive
Show threadelverkongeJul 14, 2024
@darwinwoodka @jasonkoebler even Goldman Sachs is right once in a while, though
Show threadsirlan <( rawr )Jul 13, 2024
@jasonkoebler hearthbreaking: the worst person you know made a great point
Show threadzenkatJul 13, 2024

@jasonkoebler The dot-com bubble of the 90s is the best analogy to the LLM hype. Everyone knew the Internet was going to be transformative, so massive amounts of capital rushed into the space. This drove valuations to insane levels (remember when the NASDAQ was at $5K?). It also led to huge misallocations of capital to goofy ideas. Ultimately that house of cards collapsed, but that allowed the next generation of companies to flourish and become the Internet Titans of today.

ie, Pets.com was widely ridiculed for selling dogfood over the Internet, and died in an epic flame-out ... but today it's a profitable business for Amazon.

Show threadGVF 🇨🇦Jul 17, 2024

@jasonkoebler A tech bubble?!

Have we ever seen one of those before?