One of the decisive moments in my understanding of #LLMs and their limitations was when, last autumn, @emilymbender walked me through her Thai Library thought experiment.

She's now written it up as a Medium post, and you can read it here. The value comes from really pondering the question she poses, so take the time to think about it. What would YOU do in the situation she outlines?

https://medium.com/@emilymenonbender/thought-experiment-in-the-national-library-of-thailand-f2bf761a8a83

@ct_bergstrom @emilymbender Interesting thought experiment very similar to Searle’s Chinese Room. As far as I understand that without studying it deeply, aren’t the “unlimited time”, our limitations wrt memory, and our own expectation of frustration doing too much work here?
@ct_bergstrom @emilymbender I think I get where you, Emily, are going with this, but even the question of “understanding” in a non-functional way is anthropomorphizing LLMs. It only makes sense when we try in a second step to map our knowledge about Thai signs to our existing memory of the world. But this question actually never arises for LLMs. So the only way they can understand the world is by reacting in a “right” way to inputs. But aren’t we, too?
@b3n @ct_bergstrom @emilymbender That goes back to the classic problem of rhetoric: we can't access other minds directly via text (or other modes), so must rely on hints embedded in language. That opens the ever-present possibility of deception, which we come to understand from the arc of discourse and action of an individual speaker, which reveals underlying intention. That doesn't happen with LLMs. Quite the opposite, it becomes a random walk.