Mignon Fogarty4d ago

I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.

I just talked to someone who thought AI hallucinations would be obvious because it would be obvious if you talked to a *person* who was hallucinating.

In other words, they equated "hallucination" with "sounds wacko" and accepted AI output as true because it sounded level headed.

1/2

Show threadMignon Fogarty4d ago

The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:

"Hallucination" is just a fancy word for "confidently makes mistakes":

"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"

Show threadOrion (he/him)4d ago

@grammargirl This is a good example of why that term is so dangerous. Thank you for posting it.

That said, while I have zero hope of making that term go away, we also have the word "slop" as a counter.

"Ugh. It had a hallucination..."

"Yup. And the results are now slop."

That said, I don't myself use "hallucination" in the "AI" context. I refer to the error rate, which last I checked, hovered around 40%.

Show threadDaniel ☀
@orionkidder @grammargirl I’ve heard the Spanish science communicator Ignacio Crespo argue that “hallucination” is misleading in this context, because it imports a human mental-state metaphor into a statistical text-generation error. “Confabulation” may be closer: a plausible-sounding reconstruction that fills gaps. Still, it also comes from human cognition, so it can anthropomorphise the model too.
Jun 6 at 11:11pmWeb
Show threadDaniel ☀4d ago
@orionkidder @grammargirl I think the deeper problem with “hallucination” is that it imports a human mental-state metaphor into a statistical text-generation error. That can make people expect obviously bizarre output, when the real danger is often confident, plausible-sounding falsehoods. “Confabulation” has a similar problem, though. But, I don’t know, it sounds better to me.
Show threadOrion (he/him)4d ago
@danielmunoz @grammargirl This is why I refer to its "error rate." It's a machine that produces false answers to such a large degree that it shouldn't be trusted. It's simply faulty.