Matt BlazeThe Google AI summary suggesting that people eat rocks is amusing, but it's not a great example of AI "hallucination". The text is a pretty straight and accurate summary of a satirical Onion article. This isn't a complex algorithm synthesizing bogus conclusions from good data (something that's definitely a real risk in AI systems). This is simply Google mis-categorizing non factual input as factual, something it could have (and has) done just as easily without "AI".
PlasmaGryphon@mattblaze the same was true of the ones I've seen for fighting snakes are a thesis defense, recipes for gasoline pizza and glue in pizza, and a couple others. But doesn't help that it has stripped the source and gives the impression its a synthesis of many sources when it actually just grabbed one source.
@PlasmaGryphon
I'm not saying this is *good*. I'm just saying this isn't a useful example of AI hallucination.
Google has long (and without help from AI) conflated "popular" (which the Onion certainly is) with "authoritative" (which the Onion certainly isn't).
Greg Titus@mattblaze @PlasmaGryphon Is there some specific definition of "AI hallucination" that you are referencing? Because it seems that the main difference between this case and other cases is just that it's easier to localize and identify where the algorithm picked up the wrong information.
Just because a different algorithm _also_ makes this mistake doesn't really distinguish this LLM screwup from other LLM screwups.
@gregtitus @PlasmaGryphon It's NOT an LLM screwup. It's accurately reporting the contents of the Onion piece. The problem is that the Onion piece isn't factual. The input was mislabeled from the start. This has nothing to do with how LLMs work.
@mattblaze @PlasmaGryphon In the sense that "we feed it the whole Internet and let it remix it", it is absolutely a failure of THAT algorithm.
Yes, I shouldn't have used the phrase "LLM screwup", which is vague. What I meant was "use of an LLM for a clearly inappropriate task", which is really the issue for all of these search engine "hallucinations". There is no way in which this example is any worse in appropriateness for the task than any of the other hallucinations.
David, a Bostonian in Tokyo.@mattblaze @gregtitus @PlasmaGryphon
But the original Onion piece was not mislabled: it was correctly labelled ONION, which most humans can figure out. The LLM lost that information, thus changing the label from JOKE to an implicit "I'M TELLING YOU THIS IS TRUE".
Losing that information has everything to do with how LLMs work.
This whole LLM/Chatbot thing is a plethora of incredibly stupid bad ideas, from the very basics of the underlying algorithm on up.
Jay@mattblaze @PlasmaGryphon While true, there’s a big difference between a search returning text that claims A with no clear source, simply presented as fact, and returning a link that has text that says A, where the link itself tells the story.
@WhiteCatTamer @PlasmaGryphon Which part of "I didn't say this is *good* is unclear here?
Not everything that's wrong with Google search results has to do with large language models.
@mattblaze @PlasmaGryphon I’m not saying you’re saying it’s good, I’m saying that your saying that Google could do this without AI by linking to the site is true, but there’s still important information that gets left out by doing it this way, that is, through an AI declaring it.
It’s not hallucination, but the medium is part of the message.
Steve Bellovin@mattblaze @PlasmaGryphon The problem here is that today's AI systems are really bad at understanding context, humor, sarcasm, etc. If I do a search on loose cheese on pizza and see a link to theonion.com, I know the context of what I'll get. An LLM does not, which means it should have been excluded from the training data. But can Google do that with content (like snakes at dissertation defenses) that has been reposted? I suspect that if anyone can, they can—but canthey?
Michael Richardson@mattblaze yeah, I was just trying to add that there are a wide variety of these popping up in last couple days and all the ones I looked at were the same pattern of it repeating a single search result or reddit post.
Rufus J. Cooter@mattblaze @PlasmaGryphon Right! I agree that this certainly appears to be the case; however, it also seems to raise a different problem - doesn't that make this just a garden-variety example of plagiarism?
(I thought that what AIs/LLMs do /isn't/ plagiarism, because they're synthesizing the output from a huge initial training set of data. Here, it looks like a simple copy/paste job, from a single source?)