Mastodawn

Christian Kent 𝘊𝘒 :＼﹥5d ago

Miguel Afonso Caetano

"A study of physicians in Poland who specialize in endoscopy — the use of flexible probes to examine the inside of the human body — shows how quickly AI tools can erode human abilities. The physicians, who had all performed at least 2,000 colonoscopies during their careers, were given access to an AI system that analyses colonoscopy images in real time and flags a type of precancerous intestinal lesion called an adenoma. The tool was available to the specialists on some days but not on others.

Once physicians began using it, their performance dropped significantly whenever the system was unavailable. During the three-month period before the AI tool was introduced, the specialists found at least one adenoma during 28.4% of colonoscopies. During the three-month period after the tool was introduced, the adenoma detection rate for colonoscopies performed without AI assistance decreased to 22.4%.

Gastroenterology and Hepatology, suggest that even highly skilled professionals might get worse at tasks that their job requires as they become more dependent on AI tools, says Robert Wachter, a physician at the University of California, San Francisco, who is the author of a book on how AI tools are transforming health care. The study authors say that continuous exposure to such tools can cause clinicians to become “less motivated, less focused, and less responsible when making cognitive decisions without AI assistance”.

Co-author Yuichi Mori, a physician-researcher at the University of Oslo, says that more studies are needed to confirm the phenomenon. But people who use AI tools should be aware that they risk losing some of their skills, he adds. “There is no established solution against deskilling right now. It should be a very hot research topic in the next decade.”"

https://www.nature.com/articles/d41586-026-01947-1

#AI #Deskilling #Science #Medicine

Is AI ruining our skills? Early results are in — and they’re not good

Reliance on artificial-intelligence tools degrades the abilities of physicians and software engineers, studies show.

aptitude Jun 20

@remixtures That's what happens when a glorified search engine, gets fed failing educational systems.

Randall Lee Jun 20

@remixtures No one will have the skills to conduct research on unskilling in a decade.

Sandor Spruit 🇪🇺🇳🇱🇺🇦🇨🇦Jun 20

@BLTpizza @remixtures And when we are not looking, it is not there! Problem solved! /s

König Dierk Jun 20

@BLTpizza @remixtures According to recent discussions that I had about AI, _only_ the unskilled can do research on unskilling (and you have to be drunk three days a week to talk about alcoholism)

Krzysztof Sakrejda Jun 20

@remixtures wow, apparently physicians are susceptible to the same issues with automation as pilots, nuclear power plant operators, airplane mechanics, operating engineers in general, train operators, drivers, and security guards. Shocking. I am shocked. Shocked that society's full of bros who believe anything different.

Jonathan Schofield Jun 20

@wronglang @remixtures

Yeah but “more studies are needed to confirm the phenomenon”

Krzysztof Sakrejda Jun 20

@urlyman @remixtures yeah or some people need to think about what H0 should be should be here

@wronglang @remixtures thankfully programmers, politicians and CEOs aren’t susceptible to those, though. /s

Krzysztof Sakrejda Jun 20

@unkx @remixtures would anybody notice if a CEO got replaced by a machine that makes up shit people want to hear?

CourtCantrell will not comply 5d ago

@wronglang @remixtures

"It is difficult to get a man to understand something, when his salary depends on his not understanding it."

--Upton Sinclair

Fitzibitz Jun 20

@remixtures Yep, AI lets people become sort of lazy.

Philippa Cowderoy Jun 20

@Fitzibitz @remixtures Worse: whoever made the decision that AI be used, it then forces people to "become sort of lazy"

Then their boss, who made the decision, will pass the blame on to the people using it

FossThought Jun 20

@remixtures “There is no established solution against deskilling right now..." And there's not going to be. Or if there is, it will be challenged. They want you dependent on it.

an actual bus Jun 20

@remixtures Before I form an opinion on this I want to know the number that they get *with* the AI.

If it's lower than 28.4 I will agree that this is bad. If it's higher than that I think we need to sit down and consider that maybe the question is not how do we stop AI but rather how do we take control of it from the hands of the corrupt billionaires.

WandreCanada Jun 20

@renardboy @remixtures It's not good to use AI hit rates for metrics as they often generate high false positives.

You can use clinician data because they have a proven benchmark to use as baseline.

Studies as you suggest would require years of dedicated independent research. Something that tech CEOs would never truly allow in the current climate of iron fisted control.

an actual bus Jun 20

@wandrecanada @remixtures very true, but then of course the real metric has been "number of adenomas *correctly* identified" all along. And, of course, false positives are not exclusive to AI.

My stance on AI has many nuances, but I am highly skeptical of the "it makes us stupid" narrative. What is lost in some aspects due to acquired reliance must be gained in other aspects through increased available headspace from strategic offloading.

WandreCanada Jun 20

@renardboy @remixtures I don't see an issue with taking the stance beyond that proving AI's value will take a very long time.

If half the world wasn't forcing AI into their systems to unproven efficacy results it wouldn't be an issue. However it's potential for disaster is far greater considering the heuristics of decision makers and snake oil vendors that are reaping unearned rewards.

I've been through this before with web integration. Why are we fucking up our lives with this again?

WandreCanada Jun 20

@renardboy @remixtures And for transparency's sake I was promoting web tech's potential during the time of the dot com bubble.

But I also preached to anyone seeking advice that they should understand the need before implementing. I can't tell you how many faces were surprised I told them they shouldn't hire me because they had no idea what the tech even did.

It was just FOMO and a top down directive.

EVERY

DAMN

TIME

an actual bus Jun 20

@wandrecanada @remixtures I think the dot com bubble is pretty much unquestionably the best parallel to draw with what's happening right now.

Thanks to the work of good people during that critical period, the internet is fairly free. Definitely not as much as we'd like, but also definitely more than the ruling class would like.

I believe those who will make a greater positive impact on history are those who work to democratize this new tech, not those who reject it.

@renardboy Imagine you're accustomed to foraging for food, but a scientist sticks you in a cage with a button that gives you food and water, and you quickly start relying on it. The door to the cage is opened behind you, but the button is still there, so you continue using it. It gives you a reward for minimal effort, and you are hooked. 1/3

@renardboy Now, the dispenser starts periodically giving you food that makes you sick. Instead of going back to foraging, you start trying to determine if the food is good or not via smell, or outward appearance. You don't really understand the food, food pellets weren't something you foraged for, you don't know how they're made. Suddenly, the simple task of foraging is now "press button, decide if its safe to eat, or press button again", more effort is spent for less reward 2/3

@renardboy We might tell ourselves that professionals wouldn't operate this way, but time and time again, I see it happening. I've felt it happening to me when using these tools, we've all heard about lawyers doing it. These assistants are really doing operant conditioning on us, it affects the way we think. When the GPS navigator sends you the wrong way, you don't stop using GPS or commit to planning routes in advance, you just get mad and spend time fixing it while your spatial awareness fades

an actual bus Jun 20

@caitp That is fair, and my initial reaction to your comparison was that there is no cage, but then I thought of token quotas being imposed on software developers, which despite being one myself I was lucky enough to avoid.

I do feel that this ties into the cultural context of the thing more than the thing itself though, and my point is that we can extract the latter from the former. 1/2

an actual bus Jun 20

@caitp I don't know, maybe for all my vices I have traits that make me less vulnerable than others to the pitfalls specific to llm use, but my experience with these tools has been that their capabilities are extremely narrow, that they are only capable of assisting (though meaningfully so) with part of the process of bringing ideas to fruition.

Before AI I would pace for hours on end thinking about ideas and implementations, now I pace for hours on end thinking about ideas and implementations.

Moss Wizard Jun 20

@renardboy @caitp “I don't know, maybe for all my vices I have traits that make me less vulnerable than others to the pitfalls specific to llm use”

I regret to inform you those are words only spoken by people who are *more vulnerable than others to pitfalls*, due to hubris and lack of self awareness.

an actual bus Jun 21

@Moss @caitp That is a sound argument, I suppose time will tell which one of us is right.

Michael Gemar Jun 20

@caitp @Npars01 @renardboy I’m sympathetic to such arguments, but an alternative example is you’re accustomed to foraging for food, and then a grocery store opens up near you. Very few people these days have the skill to forage for all of their nutrition, but I don’t think that’s in principle a bad thing. (1/2)

Michael Gemar Jun 20

The issues for me are whether AI-based systems are reliable, and whether they actually improve outcomes. If they don’t they shouldn’t be used, but if they do then they’re a useful tool.

(Of course there are further concerns around violation of other’s creative output with generative AI systems, but that doesn’t seem to be the case here.) (2/2)
@caitp @Npars01 @renardboy

Donald Ball Jun 20

@renardboy @wandrecanada @remixtures At least one issue is how we can continue to improve such machines if we lose the skill they’re automating.

an actual bus Jun 20

@donaldball @wandrecanada @remixtures But they're not automating innovation, they're automating duplication of effort.

Well they‘re not automating de-duplication of effort. That seems pretty sure.

@donaldball @wandrecanada @remixtures

an actual bus Jun 21

@Landa @donaldball @wandrecanada @remixtures That is for sure. That's why you still need a brain doing most of the work, and that's why you can't let yourself get dumb.

I guess it could be kind of difficult to notice when the machine one outsources thinking to degrades one‘s ability to think.

The developer of bcachefs thinks his LLM instance is conscious and in love with him, yet he seems pretty sure of his mental faculties.

ㄟ_(ツ)_ㄏ

@donaldball @wandrecanada @remixtures

David J. Atkinson 6d ago

@renardboy @wandrecanada @remixtures Exactly this. We know a reasonable baseline for human performance. How does the performance of an #AI-assisted human compare? What about an AI working alone? I also want to know about the false negatives — the missed cancer detections. We want fewer, Do missed detections go up or down with AI assistance? Are the numbers consistent over time? I have to say that these studies are at best inconclusive. The sample population (of doctors) is far too small and it is not clear that the doctors who participated in the study are representative of the population of gastroenterologist doctors.

turbulent Jun 20

@renardboy @remixtures This. Loss of skill in aircraft pilots is also an issue due to extensive use of autopilot. But nobody's calling for less usage of autopilot because the safety gain is too big to ignore. Instead training programs were adapted to make sure pilots can still react correctly during crises.

@renardboy @remixtures @wandrecanada
Yet again we meet the linguistic use of AI to mean two entirely different things
Version 1: machine learning
Version 2: large language models.

Which do we think is useful for spotting medical anomalies in diagnostic images?

an actual bus Jun 20

@OneInterestingFact @remixtures My understanding is that (if we accept that what we're dealing with qualifies as "intelligence", which I really only do to avoid derailing conversations away from their topics) LLMs are a specific application of the larger field of machine learning.

Unless I'm wrong there is relatively little to LLMs that is specific to them and not machine learning at large, but please correct me if I am.

@renardboy @remixtures

I'm afraid I don't know whether your assessment is correct. I see many potential uses for machine learning, When I look at LLMs I see theft and waste of resources.

Magnus Ahltorp Jun 20

@renardboy @OneInterestingFact @remixtures I see the big divide as being between chatbots versus classifiers. There are other things that could reasonably be called AI given how people use the term, but chatbots and classifiers are the two things that are frequently discussed in general discourse, and purposefully conflated.

degenerating degenerate Jun 20

Essentially, when "someone else does it", the specialists became "managers of the someone else that does it". It would be exactly the same if the entity they were managing was a human.

In coding too, people going into management famously have their leet skillz (should they have had any) turn to crap, since they are guiding how it is done by other now, not working at the coalface.

Since it's no longer used, the skill starts to degrade. It's normal and making space for new skills that are useful.

It's a mistake to get hung up on skills that were useful for a while and aren't any more. Sitting in front of a PC all day doing things machines now do better is not some sad loss, it's good news.

@hopeless @remixtures that would be true if the machines actually did it better.

degenerating degenerate Jun 21

@mu @remixtures You seem to be implying machines don't do "it" better... what "it" are we talking about?

Anything at all? Clearly you would be wrong.

Coding? They certainly can do it better / faster... at the same time they can produce crap if not treated how you would treat a junior on your team.

ML colon cancer matching? The article doesn't really say. Presumably if they're using it on real scans of real humans, they have some minimal experience-based faith in it.

It's good at what it's good at... generally over time those things will increase. If every single thing about AI has to be met with a sneer, eventually reality is going to intrude in an unlubed way.

@hopeless @remixtures you were the one who said "it" in your post?

I think LLMs can generate (bad) code, I don't think that's the same as writing software.

There used to be a saying "bad coders measure lines of code, good coders measure how many lines they delete, great coders measure how many lines of code they don't write" by that logic, I think LLMs are bad coders.

@hopeless @remixtures why do you think it's going to get better? It's pretty much hit a plateau for more than a year now.

Medea Vanamonde 🏳️‍⚧️Jun 20

@remixtures @lisamelton

Guillotine Jones, Flâneur Jun 20

@remixtures
You talk about clinicians losing skill through Ai use as if it's a bad thing, Miguel.
</sarcasm>

In WWII The UK needed more plane spotters. Recruits were put in a class and shown the various shapes of planes but failed in the field. They were put in the field next to an experience spotter and they were told correct or incorrect. They learned quickly.

I believe that human brain needs this reinforcement. Once it relies on tools capacity to detect patterns quickly erodes.

I’d prefer clinicians to have their own diagnosis and AI only validates their findings.

@remixtures unfortunately this gives AI bros another tool to manipulate the narrative. The playbook for AI bros will be:

Compare AI productivity to post-AI productivity (rather than pre-AI productivity) to show how "necessary" it is for professionals to "be at their best"

We have to be prepared to counter that narrative, and talking about skill erosion is something.

Making that clear connection between skill erosion and limited/rescinded availability is key. AI will not always be cheap. In fact, it's more expensive even now that most people realize.

CohenTheBlue Jun 20

@remixtures These replies are unquestionably the most varied and interesting thoughts on AI / LLMs that I've seen concentrated.

- Deskilling or respecializing in management?
https://mastodon.social/@renardboy/116782980977029184
https://mas.to/@hopeless/116782907360104119
- Foraging for food replaced with determining if sky food is not lethal, preferring not to go back to foraging (among professionals).
https://mstdn.social/@caitp/116783194421510776

- AI boom = dotcom bubble

- Skills not reinforced
https://mastodon.social/@michielw/116783214526961483
- Value, proving it
https://mastodon.social/@wandrecanada/116783029605465305

Jargoggles Jun 20

@remixtures
Taking this a step further, that when you stop doing things for yourself, you lose the ability to do it - what does that say about a certain class of people who have people do *everything* for them?

aptitude Jun 20

@jargoggles A classic empathy equation.

BoneHouseWasps🔶🇬🇧🇪🇺Jun 20

@remixtures I've been a graphic designer my whole career, pretty much. It's long been observed, in fact my first boss told me on day *one*, that you need to stay "on the boards" and he regretted that he hadn't.

Skill fade is both real and much quicker than you think.

nicole mikołajczyk Jun 20

@remixtures so apparently the main reason to use ’AI‘ is that it makes us more valuable to employers, but the main reason to not use ‘AI’ is that it makes us lose skills valuable to employers long-term. I’m not convinced by either of them.

Stevie's Yer Da'Jun 20

"are Henry Ford's assembly lines de-skilling coach builders?"

same energy and despite my huge reservations about the current private ownership and direction of AI I'm feeling that lots of AI scepticism is missing the point. It's private ownership and capitalism that are the threats as usual, not technology.

@steviesyerda @remixtures I'm not sure I get your point here. Ford didn't deskill coach builders, he didn't hire coach builders, and cars forced coach builders out of business, but didn't degrade their skills.

Did the assembly line degrade overall skills for car builders? Maybe there is something there, but I would argue that skill *increased* for the step that people were completing on the line, and that's definitely not happening for vibe-coders

Stevie's Yer Da'Jun 20

@mu @remixtures

the whole point of Fordism was to gain control of skilled labour, de-skill it onto an assembly line and thus gain control over the intensity and outputs from production. What was lost were the artisan trade skills. My comment was reflecting a similar change in the article and study. Professionals being de-skilled, potentially higher productivity using technology, but possibly a loss of quality replaced by a gain of quantity in assessments.

@steviesyerda @remixtures I mean, I think I get that, but for medical diagnosis, it's bad to reduce quality, and if you increase quantity, that multiplies the bad effect of the loss in quality.