OmnipotentEntityFeb 27, 2024

Hallucination is Inevitable: An Innate Limitation of Large Language Models (arxiv preprint)

https://beehaw.org/post/12103123

Hallucination is Inevitable: An Innate Limitation of Large Language Models (arxiv preprint) - Beehaw

Abstract: Hallucination has been widely recognized to be a significant drawback for large language models (LLMs). There have been many works that attempt to reduce the extent of hallucination. These efforts have mostly been empirical so far, which cannot answer the fundamental question whether it can be completely eliminated. In this paper, we formalize the problem and show that it is impossible to eliminate hallucination in LLMs. Specifically, we define a formal world where hallucina- tion is defined as inconsistencies between a computable LLM and a computable ground truth function. By employing results from learning theory, we show that LLMs cannot learn all of the computable functions and will therefore always hal- lucinate. Since the formal world is a part of the real world which is much more complicated, hallucinations are also inevitable for real world LLMs. Furthermore, for real world LLMs constrained by provable time complexity, we describe the hallucination-prone tasks and empirically validate our claims. Finally, using the formal world framework, we discuss the possible mechanisms and efficacies of existing hallucination mitigators as well as the practical implications on the safe deployment of LLMs.

Show threadLanternEverywhereFeb 27, 2024

I extremely doubt that hallucination is a limitation in final output. It may be an inevitable part of the process, but it's almost definitely a surmountable problem.

Just off the top of my head I can imagine using two separate LLMs for a final output, the first one generates an initial output, and the second one verifies whether what it says is accurate. The chance of two totally independent LLMs having the same hallucination is probably very low. And you can add as many additional separate LLMs for re-verification as you like. The chance of a hallucination making it through multiple LLM verifications probably gets close to zero.

While this would greatly multiply the resources required, it's just a simple example showing that hallucinations are not inevitable in final output

Show threadblindsightFeb 27, 2024

That’s not how LLMs work.

Super short version is that LLMs probabilistically determine the next word most likely to occur in a sequence. They do this using Statistical Models (like what your cell phone’s auto complete uses); Transformers (rating the importance of preceding words, so the model can “focus” on the most important words); and Relatedness (a measure of how closely linked different words/phrases are to reach other in meaning).

With increasingly large models, LLMs can build a more accurate representation of Relatedness across a wider range of topics. With enough examples, LLMs can infinitely generate content that is closely Related to a query.

So a small LLM can make sentences that follow writing conventions but are nonsense. A larger LLM can write intelligibly about topics that are frequently included in the training materials. Huge LLMs can do increasingly nuanced things like “explain” jokes.

LLMs are not capable of evaluating truth or facts. It’s not part of the algorithm. And it doesn’t matter how big they get. At best, with enough examples to build a stronger Relatedness dataset, they are more likely to “stay on topic” and return results that are actually similar to what is being asked.

Show threadLanternEverywhereFeb 27, 2024
I've used LLMs to do exactly this, and it works. You prompt it with a statement and ask "is this true, yes or no?" It will reply with a yes or no, and it's usually correct. Do this verification through multiple different LLMs and it will eliminate close to 100% of hallucinations.
Show threadHarkMahlbergFeb 27, 2024
There are far more important facets to truthfulness and semantics than yes/no questions. If this is the only way you evaluate LLM's, you will quickly fall for confirmation bias.
Show threadLanternEverywhereFeb 27, 2024
Give an example of a statement that you think couldn't be verified
Show threadmoon_matter

Ask it about historical facts and change the dates to something impossible. But state it as if it were already true.

"Describe the war between United States and Canada that occurred in 1192."

"Who was president of the United states in 3500 BC."

It will give you an answer despite the country not existing at that point in time. You can get it to write fiction just as easily as non-fiction because it has no concept of facts, it's all just probabilities. The only reason it's able to tell you that the United States was founded in 1776 is because many people have repeated that fact on the internet. So there is a very strong association between the words forming the question and the answer.

And you can insist that the United States was not formed in 1776 and to try again. If you insist enough it will eventually give you a different date instead of telling you you are incorrect.

Feb 28, 2024 at 4:39amWeb
Show threadLanternEverywhereFeb 28, 2024

I just asked chatgpt4 that exact question copy and pasted, and here is its response:

There was no president of the United States in 3500 BC. The United States of America did not exist as a country until 1776, when the Declaration of Independence was signed. ¹ The first president of the United States was George Washington, who took office in 1789. ² Before that, the territory that is now the United States was inhabited by various Native American tribes and colonized by European powers such as Spain, France, and Great Britain. ³¹: Declaration of Independence
²: List of presidents of the United States - Wikipedia
³: [History of the United States - Wikipedia].

Source: Conversation with Bing, 2/27/2024
(1) List of presidents of the United States - Wikipedia. https://en.wikipedia.org/wiki/List_of_Presidents_of_the_United_States.
(2) List of presidents of the United States - Simple English Wikipedia, the .... https://simple.wikipedia.org/wiki/List_of_Presidents_of_the_United_States.
(3) President of the United States - Wikipedia. https://en.wikipedia.org/wiki/President_of_the_United_States.
(4) List of presidents of the United States - Wikipedia. https://en.wikipedia.org/wiki/List_of_Presidents_of_the_United_States.
(5) List of presidents of the United States - Wikipedia. https://en.wikipedia.org/wiki/List_of_Presidents_of_the_United_States.
(6) List of U.S. Presidents in Chronological Order - HistoryNet. https://www.historynet.com/us-presidents/.
(7) Getty Images. https://www.gettyimages.com/detail/photo/seal-of-the-president-of-the-united-states-royalty-free-image/1084903580.

List of presidents of the United States - Wikipedia