The constant mental vigilance in a generative world is exhausting.
"I asked Claude to do $thing and it did this!"
No it didn't. No you didn't. Probably none of that happened.
And somehow, being unwilling to admit the thing is just making stuff up is annoying and unnecessary, not the damn model.
I can't get people to understand that the "hallucination" problem is unsolvable because "hallucination" is how it works. That's all it does. Next tokens based on the whole previous series of tokens that represent "the conversation" being had between prompts and responses combined with the hidden prompts that give the thing its flavor. The fact that it is "right" isn't part of it. That's why they never say, "I don't know". They don't know anything. They are literally making it up every single time. It's why they are so expensive and why they are ruining the environment. There is no recall, no memory, no "knowing". As I've seen it said elsewhere, "there is no 'there' there". It's worse than the Chinese Room thought experiment because at least that produces correct responses. This creates the illusion of a correct response. We are killing the earth and building an inescapable surveillance state around technology that will never get any better than it is right now.
It's the same "model" your know-it-all uncle uses every Thanksgiving: bloviation.
@mttaggart @jrdepriest it's just working backwards to "explain" it's bullshit answer, or so I heard.
if so that's a straight up con.
@dannotdaniel @jrdepriest Two different things.
"Reasoning" models literally talk to themselves to build out context for prompts. This leads to incredibly high token usage, and also some truly insane, neurotic-looking behavior as the model talks itself into corners.
But yes, if you ask a model "why" it did or said anything, you'll just get nonsense. There's no introspection happening there at all. Researchers can't pinpoint how some of these inferences are made, but the model itself can?
@mttaggart @jrdepriest all right well that's at least what they advertise I guess
I was impressed with the "reasoning" when they rolled it out but I've barely used it, and I've definitely encountered the situation where the thing just doesn't shut up
@mttaggart @jrdepriest neatly explains why so many managerial types are fans of it though - it sounds JUST like them
👀
@fireye @jrdepriest @mttaggart
In reality, its high dimensional vector calculus, over the trained material and your words in your context.
All this LLM workings is just vector calculus, that Leibniz devised in the 1700s. We only had the compute to do it in 2012.
Its not thinking, yet. Its not intelligence. Its a stochastic parrot trained on TBs of data, following Leibniz' dream of word-calculus.
We still dont know how consciousness works, or to create a thinking machine.
@jrdepriest @mttaggart Y'all really think it's *that* different from how humans work? People make overconfident statements about shit they don't really have 'knowledge' of all the time (see this toot/this thread/social media et al.). Sure, it's nOn-DetERMinIsTic—so are we, that's kind of the point.
Doesn't mean it's not extremely useful in certain contexts.
@zero_gravitas @jrdepriest You're arguing a point about utility I did not make. There's no doubt generative text can be "useful" in some contexts.
But the process matters here.
If you think the model's generation process is indistinguishable from human thought, we won't agree and that's that. I do think it's quite different from how humans work. If you do see a distinction, then the distinction is material when users impute the properties of one to the other.
Does the model evaluate the truth of a statement before it returns the output? Can it? If the answer is no, we have a divergence.
Yes. Is it that different.
As has been pointed out, LLMs operate using Linear Algebra Matrix Math to make predictions for "next token". That is all. Any appearance of bullshitting, bloviation, lying, confabulation, hallucinations, etc are just us anthropomorphizing math. I am guilty of it because it is much easier to talk about these systems when you do it. I'm also guilty of anthropmorphizing my car when it won't start and I beg it to please start this one time, I'll get the good gas next time. Neither system has any understanding of what is being asked of it; one of them just happens to have been trained on enough human language that it can spit out something to make it look like it does.
Is language the same as intelligence The AI industry desperately needs it to be
I had a good article that summarized historical metaphors for how the human mind works. They tend to crop up around whatever the hottest new technology is. "Mind as Steam Engine" was popular for a while. And we've all heard about keeping the humors in balance. I cant' find the article now.
There are entire books written about the idea. Our minds are not simply in our brains. Our minds are our entire body, every cell and nerve impulse, every sense and all the information that is processed back and forth. All of that is our "mind". A machine cannot create that or simulate that. LLMs aren't even built to do that. They are built to produce something that looks like our language as if we are having a conversation.
Taggart 
David Schuetz ** looking for work **
J. R. DePriest 
:EA DATA. SF:
ɹ uɐp 
Dave "Wear A Goddamn Mask" Cochran 
Mark Down
Fireye
Yet another Josh 
Hugs4friends ♾🇺🇦 🇵🇸😷
Bryan Schuetz