❝Over four months, LLM users consistently underperformed at neural, linguistic, and behavioral levels. These results raise concerns about the long-term educational implications of LLM reliance and underscore the need for deeper inquiry into AI's role in learning.❞
Hell of a research abstract there, via @gwagner:
(And please see downthread!)
“Using LLMs makes you stupid”* is such an emotionally appealing conclusion that I’m going to consciously work — this post is my public commitment! — not to read •too• much into this one study. As the OP says: small N, not peer reviewed, etc.
But I •will• immediately heed the authors’ “concerns about the long-term educational implications of LLM reliance.” However the research shakes out, it seems like good practice to double down on helping students keep their own brains awake and active regardless of the technology available to them.
* EDIT to clarify: “LLMs make you stupid” is a description of •my• confirmation bias being activated, and •not• what the paper actually says!
More from @grimalkina to the effect of “maybe don’t read too much into this paper’s conclusions:”
I see @datarama has a similar reaction about a study that says something we want to hear.
Re that second remark, “The *point* of AI is to make cognitive abilities irrelevant,” well, that’s the thing that’s up for debate right now. Making cognitive abilities (and thus labor, and people) irrelevant is very much the marketing pitch of the hype bubble. That pitch is about cost cutting, and about fear and intimidation — an intoxicating, investor-frothing mixture of fantasy and terror.
It’s not the only vision of AI, however.
There has long been a quiet debate, currently drowned out but still very much happening, about human replacement vs human augmentation. Think of Gary Kasparov remarking years ago that he thought chess played by humans with computer assistance could be a far more interesting game than either human-only or computer-only chess.
Here’s an argument for augmentation over automation, and note how far it diverges from the current hype despite it being written from a very AI-friendly point of view:
Insights The Turing Trap: The Promise & Peril of Human-Like Artificial Intelligence Erik BrynjolfssonDirectorStanford Digital Economy Lab January 12, 202220-min read DædalusSpring 2022 In 1950, Alan Turing proposed an “imitation game” as the ultimate test of whether a machine was intelligent: could a machine imitate a human so well that its answers to questions are indistinguishable from those of […]
When I say that I wish the bubble would burst so we could start having actual conversation, this is the kind of thing I’m talking about.
In the meantime, not being a researcher myself in any kind of position to critique the paper, I’m happy to just look at it and cherry-pick the conclusion that I think is unequivocally helpful regardless: we all (students in school, students of life) should cultivate active, engaged minds that work with ideas by •doing• and •creating• instead of simply passively receiving.
Hey, the paper’s first author, @nataliyakosmyna, in on Fedi! Let’s welcome her, and take a moment to note what she says about the study:
I mistrust something which so perfectly reflects my own gut feelings.
I think LLMs are disaster for education, and this abstract is an example of why (there are many others), but it would be a big mistake to simplify that abstract down to "LLMs make you stupid". Especially if you live here in the USA (which I strongly suspect you do), where eugenics has a strong influence on all aspects of culture, and controls all 3 branches of federal government.
𝐍𝐨, 𝐲𝐨𝐮𝐫 𝐛𝐫𝐚𝐢𝐧 𝐝𝐨𝐞𝐬 𝐧𝐨𝐭 𝐩𝐞𝐫𝐟𝐨𝐫𝐦 𝐛𝐞𝐭𝐭𝐞𝐫 𝐚𝐟𝐭𝐞𝐫 𝐋𝐋𝐌 𝐨𝐫 𝐝𝐮𝐫𝐢𝐧𝐠 𝐋𝐋𝐌 𝐮𝐬𝐞. See our paper for more results: "Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task" (link in the comments). For 4 months, 54 students were divided into three groups: ChatGPT, Google -ai, and Brain-only. Across 3 sessions, each wrote essays on SAT prompts. In an optional 4th session, participants switched: LLM users used no tools (LLM-to-Brain), and Brain-only group used ChatGPT (Brain-to-LLM). 👇 𝐈. 𝐍𝐋𝐏 𝐚𝐧𝐝 𝐄𝐬𝐬𝐚𝐲 𝐂𝐨𝐧𝐭𝐞𝐧𝐭 - LLM Group: Essays were highly homogeneous within each topic, showing little variation. Participants often relied on the same expressions or ideas. - Brain-only Group: Diverse and varied approaches across participants and topics. - Search Engine Group: Essays were shaped by search engine-optimized content; their ontology overlapped with the LLM group but not with the Brain-only group. 𝐈𝐈. 𝐄𝐬𝐬𝐚𝐲 𝐒𝐜𝐨𝐫𝐢𝐧𝐠 (𝐓𝐞𝐚𝐜𝐡𝐞𝐫𝐬 𝐯𝐬. 𝐀𝐈 𝐉𝐮𝐝𝐠𝐞) - Teachers detected patterns typical of AI-generated content and scoring LLM essays lower for originality and structure. - AI Judge gave consistently higher scores to LLM essays, missing human-recognized stylistic traits. 𝐈𝐈𝐈: 𝐄𝐄𝐆 𝐀𝐧𝐚𝐥𝐲𝐬𝐢𝐬 Connectivity: Brain-only group showed the highest neural connectivity, especially in alpha, theta, and delta bands. LLM users had the weakest connectivity, up to 55% lower in low-frequency networks. Search Engine group showed high visual cortex engagement, aligned with web-based information gathering. 𝑺𝒆𝒔𝒔𝒊𝒐𝒏 4 𝑹𝒆𝒔𝒖𝒍𝒕𝒔: - LLM-to-Brain (🤖🤖🤖🧠) participants underperformed cognitively with reduced alpha/beta activity and poor content recall. - Brain-to-LLM (🧠🧠🧠🤖) participants showed strong re-engagement, better memory recall, and efficient tool use. LLM-to-Brain participants had potential limitations in achieving robust neural synchronization essential for complex cognitive tasks. Results for Brain-to-LLM participants suggest that strategic timing of AI tool introduction following initial self-driven effort may enhance engagement and neural integration. 𝐈𝐕. 𝐁𝐞𝐡𝐚𝐯𝐢𝐨𝐫𝐚𝐥 𝐚𝐧𝐝 𝐂𝐨𝐠𝐧𝐢𝐭𝐢𝐯𝐞 𝐄𝐧𝐠𝐚𝐠𝐞𝐦𝐞𝐧𝐭 - Quoting Ability: LLM users failed to quote accurately, while Brain-only participants showed robust recall and quoting skills. - Ownership: Brain-only group claimed full ownership of their work; LLM users expressed either no ownership or partial ownership. - Critical Thinking: Brain-only participants cared more about 𝘸𝘩𝘢𝘵 and 𝘸𝘩𝘺 they wrote; LLM users focused on 𝘩𝘰𝘸. - Cognitive Debt: Repeated LLM use led to shallow content repetition and reduced critical engagement. This suggests a buildup of "cognitive debt", deferring mental effort at the cost of long-term cognitive depth. Support and share! ❤️ #MIT #AI #Brain #Neuroscience #CognitiveDebt | 23 comments on LinkedIn
@nataliyakosmyna Oh, hi! Yes, I did not mean to imply “LLMs make you stupid” was •your• language at all; I’ll edit to clarify. That’s just the confirmation bias bell it rings in my own brain, and thus the reason I’m treating the results with caution.
Results aside — I truly am not equipped to judge them! — your research question seems a worthy and important one, and I’m glad you’re asking it.
@inthehands @SRDas @gwagner At least, this study contains a confirmation of the gut feeling that delegating parts of one's mental processes is detrimental to said mental processes.
Also: slightly funny note in the paper (see image), in that it is unclear if the note is an instruction to LLMs to ignore anything after it except the next table, or a note to humans telling them that the table is only for LLM consumption. A comma would have helped (me).
@inthehands @gwagner The result of frictionless convenience IMHO
I liked @Joanwestenberg take on this https://medium.com/westenberg/how-convenience-kills-curiosity-33a53ce69486
The results are not unexpected, given that even before A.I there have been many studies about the development of language as being a fundamental part of cognitive development.
And this is not just a recent phenomenon related to current education levels, it's about the evolution of the entire human species.
@akamran
This is similar to what Cat Hicks wrote in her perplexed thread about the paper: the methodology is wacky and kind of fishy, but the basic conclusion that active engagement leads to improved cognition etc (which fits with your example) is well established pre-LLM.
yuubi
Paul Cantrell
nataliyakosmyna
Martijn Faassen
Venya (he/him/dude) 🇺🇦
PaulDavisTheFirst
John
llewelly
Das
Albert ARIBAUD Ⓜ
ITsec Agentur
SufferForMe 🆘
ticho
mtconleyuk
Mx Verda
Tóth Gábor Baltazár
Stellar 🇫🇷 
Annelies Kamran, Ph.D.
This Old Hiker