Mastodawn

How AI Impacts Skill Formation:

"We conduct randomized experiments to study how developers gained mastery of a new asynchronous programming library with and without the assistance of AI.

We find that AI use impairs conceptual understanding, code reading, and debugging abilities, without delivering significant efficiency gains on average. Participants who fully delegated coding tasks showed some productivity improvements, but at the cost of learning the library."
🤔
https://arxiv.org/abs/2601.20245
1/n

Show thread

AkaSci 🛰️1d ago

Related article - "Is AI ruining our skills? Early results are in — and they’re not good"

"Evidence suggests that AI-driven ‘deskilling’ is starting to happen in medicine, computer science and other fields. Researchers are now discussing how to preserve important human expertise in the age of AI."

https://www.nature.com/articles/d41586-026-01947-1

Is AI ruining our skills? Early results are in — and they’re not good

Reliance on artificial-intelligence tools degrades the abilities of physicians and software engineers, studies show.

Show thread

AkaSci 🛰️1d ago

"Participants were divided into 3 groups: LLM, Search Engine, and Brain-only (no tools).

EEG revealed significant differences in brain connectivity: Brain-only participants exhibited the strongest, most distributed networks; Search Engine users showed moderate engagement; and LLM users displayed the weakest connectivity.

The LLM group's participants performed worse than their counterparts in the Brain-only group at all levels: neural, linguistic, scoring."

https://arxiv.org/abs/2506.08872
3/n

Show thread

⁡jaseg 23h ago

@AkaSci I had a look at this study when it came out last year, and it looked very shaky scientifically. I can’t find the thread anymore, but I remember finding that this study doesn’t support any of the conclusions people usually draw from it.

Show thread

⁡jaseg

@AkaSci IIRC the most basic problem is one of people mid-interpreting differences in EEG activity as a good or a bad thing. EEG is a really coarse tool and doesn’t allow for conclusions like that. The premise that EEG allows any measurement of “brain connectivity” is also shaky at best and bullshit at worst, depending on how far you’re willing to stretch that “connectivity” term.

Show thread

⁡jaseg 23h ago

@AkaSci IIRC (I might be wrong, it has been a while, please read for yourself) they just did a basic correlation analysis between electrode signals which doesn’t really tell you anything of value.

Show thread

⁡jaseg 23h ago

@AkaSci Like on other social media, over here on masto, we need to be careful not to hype, misinterpret, or overstretch scientific results just because they happen to align with our biases or worldview.

Show thread

⁡jaseg 23h ago

@AkaSci One more note on that article: The ArXiv version is a preprint that hasn’t passed peer review, and AFAICT today, a year after its initial publication the article still hasn’t been accepted in peer review and properly published anywhere. This suggests to me that they are having trouble convincing peer reviewers of the merits of that article.