Taylor Webb

@[email protected]
20 Followers
60 Following
22 Posts
Studying cognition in humans and machines
Taylor WebbJan 11, 2023
Melanie MitchellJan 11, 2023
New blog post: @taylorwwebb, co-author of fascinating paper on GPT-3's analogy-making ability, responds to my perspective:
https://aiguide.substack.com/p/response-to-on-analogy-making-in
Response to "On Analogy-Making in Large Language Models"

My first post on this blog was a perspective on a very interesting paper by Webb et al., “Emergent Analogical Reasoning in Large Language Models”. Taylor Webb, the first author of that paper, responded by email to my perspective, and he gave me permission to post his response on this blog. Here is Taylor’s thoughtful response, which is well worth reading. I will make a few comments at the end.

AI: A Guide for Thinking Humans
Taylor WebbJan 5, 2023
Jascha AchterbergJan 5, 2023

What are your favourite projects investigating if / how different large #foundational #models do #logical #reasoning ? Or how their "next token prediction mechanism" emulates reasoning.

Still trying to make my mind up whether the internal dynamics of these models are worth investigating.

Very curious to hear people's thoughts!

#NLProc #LLM #genAI #AI #ML #logic @cogsci @cognition @neuroscience #neuroscience #cognition

Taylor WebbJan 4, 2023
Matt SlocombeJan 3, 2023

📢 📢 Delighted to announce that the Analogical Minds Seminar is returning next week with a new series of talks on analogical processes in cognition and learning.

Over the spring term, we’ll be covering topics such as reasoning, language, development, conceptual blending, mathematics, design, and science education. Click on the image below for the full spring programme.

Registration and further info: http://www.analogicalminds.com

All welcome!

@cognition #psychology #development #education #AI

Analogy List - Analogical Minds Seminar

Analogical Minds Seminar Analogical Minds is a weekly online seminar dedicated to exploring the role of analogy, metaphor, and relational processes in cognition and learning. Our speakers discuss research from a wide range of disciplinary perspectives, including cognitive and developmental

Taylor WebbJan 2, 2023
David BauDec 21, 2022

I want to show the NSF there would be broad support+utility for a "National Deep Inference" service for >100b LLMs.

If your research would be enabled by an inference service on open LLMs w API access+overrides to internal activations, params, gradients: please boost this thread!

(I'm also gathering feedback on twitter - more details here:)

https://twitter.com/davidbau/status/1605609105824964611

David Bau on Twitter

“I want to show the NSF there would be broad support+utility for a "National Deep Inference" service for >100b LLMs. If your research would be enabled by an inference service on open LLMs w API access+overrides to internal activations, params, gradients: Please Like this thread!”

Twitter
Taylor WebbDec 21, 2022
We also took a look at some other analogy tasks, including tasks that involved more complex, naturalistic relations. These included four-term verbal analogies, where GPT-3 matched human performance, and a classic problem-solving task (Duncker's radiation problem), in which a solution can be discovered by making an analogy between the problem and a previously presented story.
Show threadTaylor WebbDec 21, 2022
Given the recent debates surrounding the reasoning abilities of LLMs, we wondered whether they might be capable of this kind of zero-shot analogical reasoning, and how their performance would stack up against human participants.
Show threadTaylor WebbDec 21, 2022
Analogical reasoning is often viewed as the quintessential example of the human capacity for abstraction and generalization, allowing us to approach novel problems *zero-shot*, by comparing them to more familiar situations.
Taylor WebbDec 21, 2022

Here's something to kick things off over here: in a new paper, we found that GPT-3 matches or exceeds human performance on zero-shot analogical reasoning, including on a text-based version of Raven's Progressive Matrices.

https://arxiv.org/abs/2212.09196v1

Emergent Analogical Reasoning in Large Language Models

The recent advent of large language models has reinvigorated debate over whether human cognitive capacities might emerge in such generic models given sufficient training data. Of particular interest is the ability of these models to reason about novel problems zero-shot, without any direct training. In human cognition, this capacity is closely tied to an ability to reason by analogy. Here, we performed a direct comparison between human reasoners and a large language model (the text-davinci-003 variant of GPT-3) on a range of analogical tasks, including a novel text-based matrix reasoning task closely modeled on Raven's Progressive Matrices. We found that GPT-3 displayed a surprisingly strong capacity for abstract pattern induction, matching or even surpassing human capabilities in most settings. Our results indicate that large language models such as GPT-3 have acquired an emergent ability to find zero-shot solutions to a broad range of analogy problems.

arXiv.org