I have been away from Mastodon for a while, not finding the time to be active here.
In any case here goes the final version of a paper on content and meaning in LLMs, co-authored with @raphaelmilliere. It lived a long preprint life, but is now published in an outstanding special issue of the journal Philosophy and the Mind Sciences.
The Vector Grounding Problem - https://philosophymindscience.org/index.php/phimisci/article/view/12307
The whole issue is worth checking out!
#philAI #llm
The Vector Grounding Problem
| Philosophy and the Mind Sciences
Philosophy and the Mind Sciences (PhiMiSci) focuses on the interface between philosophy of mind, psychology, and cognitive neuroscience. PhiMiSci is a peer-reviewed, not-for-profit open-access journal that is free for authors and readers.
A personal update -- I'm happy to share that I'll be joining Oxford this fall as an associate professor, as well as a fellow of Jesus College and affiliate with the Institute for Ethics in AI. I'll also be establishing my AI2050 Fellowship from Schmidt Sciences there. Looking forward to getting started!
There's a lot more in the full paper – here's the open access link:
https://www.sciencedirect.com/science/article/pii/S0749596X25000695
Special thanks to Taylor Webb and Melanie Mitchell for comments on previous versions of the paper!
9/9
This opens intesting avenues for future work. By using causal intervention methods with open-weights models, we can start to reverse-engineer these emergent analogical abilities and compare the discovered mecanisms to computational models of analogical reasoning. 8/9
But models also showed different sensitivities than humans. For example, top LLMs were more affected by permuting the order of examples and were more distracted by irrelevant semantic information, hinting at different underlying mechanisms. 7/9
We found that the best-performing LLMs match human performance across many of our challenging new tasks. This provides evidence that sophisticated analogical reasoning can emerge from domain-general learning, where existing computational models fall short. 6/9
In our second study, we highlighted the role of semantic content. Here, the task required identifying specific properties of concepts (e.g., "Is it a mammal?", "How many legs does it have?") and mapping them to features of the symbol strings. 5/9
In our first study, we tested whether LLMs could map semantic relationships between concepts to symbolic patterns. We included controls such as permuting the order of examples or adding semantic distractors to test for robustness and content effects (see full list below). 4/9
We tested humans & LLMs on analogical reasoning tasks that involve flexible re-representation. We strived to apply best practices from cognitive science – designing novel tasks to avoid data contamination, including careful controls, and doing proper statistical analysis. 3/9
We focus on an important feature of analogical reasoning often called "re-representation" – the ability to dynamically select which features of analogs matter to make sense of the analogy (e.g. if one analog is "horse", which properties of horses does the analogy rely on?). 2/9
Dimitri Coelho Mollo