Baldur BjarnasonSep 23, 2023

“Elegant and powerful new result that seriously undermines large language models”

Like I’ve been saying for a while now: LLMs do not think or reason. They are not on the path to AGI. They are extremely limited correlation and text synthesis machines. https://garymarcus.substack.com/p/elegant-and-powerful-new-result-that

Elegant and powerful new result that seriously undermines large language models

Wowed by a new paper I just read and wish I had thought to write myself. Lukas Berglund and others, led by Owain Evans, asked a simple, powerful, elegant question: can LLMs trained on A is B infer automatically that B is A? The shocking (yet, in historical context, see below, unsurprising) answer is no:

Marcus on AI
Show thread418 I'm a TeapotSep 23, 2023
@baldur unfortunately, this result is rather weak. it basically says, "by providing insufficient training data, the neural network fails to generalise". they taught the NNs that "A is B" and it did not magically know that "B is A". if, however, they had also taught it deduction rules, it would have done better. this can be easily verified by giving a trained NN some deduction rules, "facts" with made up words, and queries about these facts.
Show thread418 I'm a TeapotSep 23, 2023
@baldur i agree that these techniques are unlikely to lead to AGI, and there are plenty of reasons to object to and be sceptical about the current LLM vogue, but they are not so unsophisticated that they fail the most basic of tests.
Show thread418 I'm a Teapot
@baldur indeed, if they had trained it on a corpus of “A implies B” rather than “is” it would be incorrect to deduce that “B implies A”. So it’s not even obvious that what they have found is a defect.
Sep 23, 2023 at 2:51pmWeb