RE: https://mathstodon.xyz/@gregeganSF/116606390500783662

I still don't understand why anyone uses AI. Under what circumstances is this level of untrustworthiness acceptable?

@NFG Well, this example goes all the way back to Babbage.

"Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?"

He's adding 4000 inputs to a database of billions, and wondering why the machine isn't only considering his inputs.

People genuinely think LLMs are magic.

@tursilion yeah, I mean I know why it failed, but it should be an instructional experience for any person who expects this sort of thing to work.

It's not just not magic, it's a vast wilderness of random inputs and scrambled outputs without meaningful guardrails.

A total bullshit machine that should never have been allowed to survive.

@NFG It's only gonna get worse.

I am ambivalent about the machine itself, honestly. It's interesting from a machine point of view and predictable as the next step. So were neural nets that could steer a car or play Pong, and predictive algorithms that could solve curves and converge complex mathematics. I love using a computer's speed to brute force a problem, or genetic algorithms that attempt to evolve to an answer.

We've all been living with machine learning for decades now. Dictation software, OCR, game theory, none of it is new and it's in every home.

But because LLMs resemble /reasoning/, they've been blown way out of proportion. But they don't reason. It's a trick. It's Eliza on steroids. And that is ruining everything as people with money to make are successfully claiming it's the answer to everything.

And wow do I sound like the guy in the sci-fi stories arguing against robot sentience. Oh well. It's true for now.

Sadly I cant post my quote on his page without signing up.