Baldur BjarnasonSep 23, 2023

“Elegant and powerful new result that seriously undermines large language models”

Like I’ve been saying for a while now: LLMs do not think or reason. They are not on the path to AGI. They are extremely limited correlation and text synthesis machines. https://garymarcus.substack.com/p/elegant-and-powerful-new-result-that

Elegant and powerful new result that seriously undermines large language models

Wowed by a new paper I just read and wish I had thought to write myself. Lukas Berglund and others, led by Owain Evans, asked a simple, powerful, elegant question: can LLMs trained on A is B infer automatically that B is A? The shocking (yet, in historical context, see below, unsurprising) answer is no:

Marcus on AI
Show threadSlightlyCyberpunkSep 23, 2023

@baldur Always love seeing a dose of reality to counter the "AI" hype! I do kinda wonder though...this does not feel entirely different from how human minds often work. If I hear a name outside of the typical context I might be sitting there for *days* going "I know I know that name...why do I know that name??" until I find that one specific piece of context to attach to that name which makes everything fall into place...

It's not a database where you can query for some string and any related facts just pop right out...but we already have those. I suspect that is not a necessary part of being able to think or reason and could potentially even be detrimental.

Show threadBornach

@admin @baldur

Not sure how context can be a factor in the experiment that the authors of the paper tried

https://owainevans.github.io/reversal_curse.pdf

They tried training LLMs using facts stated in either direction. The LLM was much better at answering the question if it was given in the same direction as the facts it was trained on

Sep 23, 2023 at 6:11pmWeb
Show threadSlightlyCyberpunkSep 23, 2023

@bornach @baldur The context is the thing that allows me to make the connection in the right direction.

I might see someone walking down the street and think they look familiar but not be able to say who they are. If I get a call the next day from my doctor's office, I might remember that the person I saw was the doctor's receptionist. If I start from thinking about the doctor, I can remember the people I interact with there; but I can't necessarily make the same connection going the other way.