Mignon Fogarty4d ago

I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.

I just talked to someone who thought AI hallucinations would be obvious because it would be obvious if you talked to a *person* who was hallucinating.

In other words, they equated "hallucination" with "sounds wacko" and accepted AI output as true because it sounded level headed.

1/2

Show threadMignon Fogarty4d ago

The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:

"Hallucination" is just a fancy word for "confidently makes mistakes":

"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"

Show threadOrion (he/him)4d ago

@grammargirl This is a good example of why that term is so dangerous. Thank you for posting it.

That said, while I have zero hope of making that term go away, we also have the word "slop" as a counter.

"Ugh. It had a hallucination..."

"Yup. And the results are now slop."

That said, I don't myself use "hallucination" in the "AI" context. I refer to the error rate, which last I checked, hovered around 40%.

Show threadAccordionBruce4d ago
@orionkidder @grammargirl
The explanation has to include that if you believe what the AI tells you then you are hallucinating
Show thread2xfo4d ago
@AccordionBruce @orionkidder @grammargirl
Exactly this.
Hallucination is an act of cognition. The machine doesn't
Show threadgotofritz4d ago

@RnDanger @AccordionBruce @orionkidder @grammargirl

It seems such a pointless, minor nuance that will make no difference whatsoever in practice 😅

(yes I am aware talking about this kind of minor nuances is your day job, but still, someone's gotta say it)

Show threadElf Burgerman4d ago
@gotofritz @RnDanger @AccordionBruce @orionkidder @grammargirl
Language can be used as one of the most dangerous tools we have because it shapes the way we think (and thus our future) mostly on a subconscious level. The more subtly a word misleads, the more difference it can make in practice.
Show thread2xfo4d ago
@elfburgerman @gotofritz @AccordionBruce @orionkidder @grammargirl
I agree.
"Hallucination" is a great marketing term to make people want to trust a machine, but it's a pretty poor choice of words to convey any understanding of what the machine does or how it does it
Show threadOrion (he/him)4d ago
@RnDanger @elfburgerman @gotofritz @AccordionBruce @grammargirl Exactly. Making machines seem like magic, seem like they have no internal mechanism, is a common tactic. It's why we refer to external hard drives that we don't own or control as "the cloud."
Show threadgotofritz

@orionkidder @RnDanger @elfburgerman @AccordionBruce @grammargirl

Sounds all a bit conspiracy theory to me.

There is nothing positive about "hallucinating", I wouldn't ride a bus if I knew the driver was prone to hallucinating

Jun 7 at 1:32amWeb
Show threadOrion (he/him)4d ago

@gotofritz @RnDanger @elfburgerman @AccordionBruce @grammargirl It's a marketing tactic.

And the problem with the metaphor of hallucination was explained at the top of the thread.

I'll be blocking you if you keep playing ignorant.