Oh the power plant problem is one i love to point out too. A 100 watt brain vs a gigga watt computer cluster.
It's what i said when IBMs Watson did that game show nonsense. Require Watson to be in the room running on their own mobile power source and have it beating people at quizzes then we'll talk.
#LLM are brute-forcing their way through absurd amounts of data to generate an autocomplete output for any given input that approximates outputs a human might give instead. They lack a few distinct properties of human cognition, including language, that more brute force alone cannot compensate for. Because they can only ever internalize and compute *intra*textual context. Incidentally, humans need much less input(!) to learn language. Probably because they can contextualize across domains. 🧵
My experience with LLMs and my understanding of their architecture (granted, have not engaged past summer last year) would say otherwise: These models can only reproduce metaphorical patterns that are part of their training.
Also they don't attempt to mimic an algorithmic model of the human lexicon at all. If you want to look into knowledge representation bound to a syntax interface, something like Barsalou Frames seems much more closely aligned with human cognition.
True. There are a lot of metaphors in the training data, so it's likely just emulating us. Can you suggest one that LLMs fail at?
I've noticed it sucks at CSPs that it's not seen. (Probably due to the feedforward structure)
I don't know about Barsalou Frames, will read up on it!
It's not "likely", regurgitating training patterns through a fuzzy weighted autocomplete is the whole architecture. There is no emulation of human cognition.
The output quite reliably fails for ad-hoc metaphors. Worse still: when you need a shared origo to decode the metaphor, even for discourse deixis, that's simply inaccessible to a model.
Convincing models usually are designed to detect prompts that lead to user frustration and fall back to repair questions, btw.
Jyoti Mishra
Martin Owens 
jakob.pxi
Tim Hanson