Chris Hanson@siracusa I was really disappointed to hear you say, shortly before 1hr40m in the latest released episode of ATP, that LLMs are “really good at understanding” which is an enormous category error. They don’t *understand* anything! What they’re good at is *using statistics to predict what text could plausibly come next*.
Damien Sorresso@enhancedscurry @siracusa Yes, but I don’t think it goes the other direction—being able to make predictions about something doesn’t necessarily mean understanding it.
@eschaton @siracusa But if you have a system which can make predictions about arbitrary subjects with sufficient training, why does the training mechanism matter? It really gets into "quacks like a duck" territory. Not there yet, mind you, but I don't think a reliance on statistics is all that different from we do things, which is basically pattern recognition.
Sören@enhancedscurry @eschaton @siracusa I’m not an expert, but these don’t feel like something sufficiently good models will fix:
• hands with too many or too few fingers
• distortions in image enhancements
• ChatGPT guessing how many letters a word has
These are all “human looks at it for mere seconds and seems the error” mistakes that such a model doesn’t seem to grok
@chucker @enhancedscurry @siracusa Exactly; while I think statistical prediction models can get really good, ultimately they’re going to be limited by the lack of knowledge unless the networks themselves happen to develop an underlying understanding as a side-effect of training.
Lennvor@siracusa @enhancedscurry @chucker
Considering LLMs has made me reflect on how our own brain has separate modules for "understanding" (predicting, modelling) different things, such that not only is our understanding not reducible to language, the very notion of "meaning" for a human appeals to those non-linguistic models. Language production & understanding for us consists of converting nonverbal internal states/model into language and back. Of course there are complex feedbacks where language influences nonverbal understanding but it still involves separate systems and understanding doesn't reduce to language.
What I'm not sure of is whether this is an inherent feature of anything that could act on its "understanding" as well as we do (which LLMs currently can't, whatever label we assign to their internal processes), or whether it's just the way we happen to do it and a system without such a separation could perform as well as we do. I can't help feeling it's an inherent feature but I haven't found a logical justification for it.