“Over four months, LLM users consistently underperformed at neural, linguistic, and behavioral levels.”

https://www.media.mit.edu/publications/your-brain-on-chatgpt/

Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task – MIT Media Lab

 This study explores the neural and behavioral consequences of LLM-assisted essay writing. Participants were divided into three groups: LLM, Search Engine…

MIT Media Lab

Microsoft researchers say an overdependency on AI tools like Copilot negatively impacts people's critical thinking capabilities.

https://www.windowscentral.com/software-apps/copilot-and-chatgpt-makes-you-dumb-new-microsoft-study

Will an overreliance on Copilot and ChatGPT make you dumb? A new Microsoft study says AI 'atrophies' critical thinking: "I already feel like I have lost some brain cells."

Microsoft researchers say an overdependency on AI tools like Copilot negatively impacts people's critical thinking capabilities.

Windows Central
I’m refusing to on board to Copilot at work. I’m the only person. When asked why I sent them the MIT and Microsoft research papers as openers.
@GossiTheDog AI is OK for me as long it stays in a Browser Window.
@masek @GossiTheDog that isn't necessarily meaningful given the constant attempts by Google to further enshittify the Web by pushing intrusive, unfree technologies. As long as people are using Chrome, it's only a matter of time until whatever you think is safer in a browser will be just as intrusive if not more than a standalone software application.
@unexpectedteapot @masek @GossiTheDog So, teach people to use better browsers and better search engines.

@masek
And that page in that browser window is served is served from a fossil fuel-powered, dinking water-guzzling DC near you.

@GossiTheDog

@dzwiedziu @masek @GossiTheDog and so is all the internet traffic ever. what is your point?

@lukerufkahr
The Internet is occasionally beneficially and runs on more or less efficient infrastructure.

The slop extruders run inefficient algorithms and add up to that cost exponentially, providing no benefits to the general society.

@masek @GossiTheDog

@dzwiedziu @lukerufkahr @masek @GossiTheDog this. Traditional general purpose compute is pretty damn efficient at this point. AI simply is not.

Just SWAG the wattage needed to serve an old fashioned Google search query, then the wattage for a Gemini AI inference response to the same query. At least an order of magnitude.

And how many Google queries per day globally?

🌎🔫

@masek @GossiTheDog it’s ok with me as long as it stays verbal. As an intangible prompt it works well to point you in a new direction. The output is too inconsistent for a physical form, that makes it too easy to mistake for information. Outside of that it’s the friend that everyone knows is a liar. Crazy stories at the bar but I’d never work with the dude.
@masek @GossiTheDog lemme correct that for you because you misspelled "AI is okay for me because it's actively killing the planet and using up valuable fresh water and I'm fine with the dehumanization of life".

@jadedtwin Tha AI is running on Mac Mini M2 Pro at home. So I can benchmark exactly what it needs. A text query is usually about 300 Ws of electricity, there is no water involved.

About 85% of my current electricity usage is self-produced by my photovoltaics.

It doesn't help you making a good argument by proclaiming the other side an idiot that doesn't understand what he is doing.

@GossiTheDog

@masek @jadedtwin @GossiTheDog but what about the training costs, they ask as a large percentage of the world is bombed into oblivion and tankers are set on fire.

Thank fuck we recycle and scold each other onlne, changing the world by making ourselves miserable one toot at a time.

@masek @jadedtwin @GossiTheDog As those numbers would be interesting, indeed. Are you sure you mean energy consumption of 300 Ws (Watt seconds = Joule), because 300 J ≈ 0.083 Wh. That would be extremely low. That's only about 3 % of the estimated 2.5 Wh per prompt to ChatGPT. Maybe you miscalculated it?

@camelCaseNick @jadedtwin @GossiTheDog Nope: the wattage increases by 5-10W during the query which takes about 30s. The numbers reported in the press are very likely inaccurate.

I use ollama on a Mac Mini M2 Pro.

@masek i think those media reports probably factor in training data. Also i wouldn't be surprised if chatgpt consumed more power than local models on Apple due to the model size and gpu architecture
@masek self-delusion is a dangerous slippery slope. "I won't get addicted to heroin" says every heroin addict in the beginning
@jonahgibberish I am 60 now. I think I don't have the energy for a new addiction any more 😄

@masek @GossiTheDog

But it won't ;-). You'll be reading and it will rewire your Brain...

@glitzersachen When I started with computers in the late 70s they prophetised me square eyes ...

@masek

I (also) was only half serious. But I also was only half joking, too.

"The Shallows: What the Internet Is Doing to Our Brains" by Nicolas Carr has some interesting ideas on (a) the cultural role of reading and writing and (b) the constant rewiring occurring in human brains.

We know that LLMs suffer model collapse when trained with the output of other LLMs. Given those two data points --- humans cannot firewall input effectively, their brains costantly rewire, and that the output of LLMs is somehow demonstrably different from text created by humans, even if we don't understand why and how --- I am somewhat weary of letting LLM output into my brain.

I know, this sounds radical, but haven't all experienced already how bad code or badly written copy/text by others scramble our brains so we're either exhausted or find ourselves incapable of producing good text ourselves for some time.

I fear a similar effect, only more insidious and permanent. Those two papers seem to confirm my fears.

@GossiTheDog same, got many replies (quite fast!) with arguments on what those papers are missing and why they are so irrelevant. lol.
@GossiTheDog I gave it the benefit of the doubt. It sucked at the first questions I asked it, ie generated code that didn't compile, that was impossible even according to the documentation. All other reasons aside that alone did it for me.

@GossiTheDog it feels like the equivalent to hand-holding. A decade ago when I first tried to mentor someone, I noticed that offering quick replies to problems they could figure out on their own, made them stop thinking.

Ignoring them for 15-30 minutes before replying usually resulted in a:

"never mind, I figured it out".

Feels the same when using Copilot. I tend to disable it.

@gabriel @tiotasram @GossiTheDog
A maxim teachers sometimes use: “the minimum intervention to get them unstuck”

(Not always the right advice, but it is more often than our natural inclinations would say it is!)

@inthehands @gabriel @GossiTheDog yup.

The Socratic method is great for avoiding over-help too.

@inthehands @tiotasram @GossiTheDog yes. I only give prompt answers when I know it's something that they can't know in their current learning curve. But things that have been explained already that just need to be put together to form a solution, I tend to leave them to it for a while before asking some guiding questions. And eventually help unblock if they are still stuck.

@GossiTheDog Same situation (but we're two) and we have sent those same docs + the Apple and the recent EchoLeak vuln.

Perceived as negative..

@GossiTheDog I woke up and logged into work and magically had it enabled and added to a teams group. I have to refrain from shit posting.
@GossiTheDog
Think I've used an LLM for actual work purposes twice so far. Most of the time just use it to see if the hidden prompt I add to documents works well enough to mess with anyone trying to feed them to the system...
@GossiTheDog what does "on boarding" to copilot mean here? Enabling it or usage or...?
@GossiTheDog I'm dealing with a coworker who is just uncritically copying and pasting from CoPilot or Gemini or something. They're not checking to make sure the commands and options-flags actually exist, and don't understand why I'm getting frustrated with them.
@drsbaitso @GossiTheDog These people should be let go, ASAP; they're not providing any kind of value to the organization if they can't even double check their work.
@GossiTheDog Call me cynical, but I think lots of companies would be just fine if AI leads to individual skill attrition in the long term. They could reap, as they see it, short-term productivity gains at the cost of "expending" a human resource that they can then lay off. Not all companies, but many.

@GossiTheDog

As will I, from now on.

@GossiTheDog Copilot sucks. I suggest aider or cline instead
@GossiTheDog hi, could you send me the papers you mentionned, so i can also promote awareness 😉
@GossiTheDog so, if over-reliance makes you dumb, what does simple usage make you?

@jb @GossiTheDog The paper covers it. They had the brain-only group use ChatGPT in a subsequent session, and noticed no significant change in neural activity.

So if you do the work first, then the LLM has no adverse effects. But if you've already done the work, then what's even the point of using it?

@richarddegenne @GossiTheDog

But that’s not even usage, then. You’ve solved the problem already.

@GossiTheDog
@nadia_z
Research from Captain Obvious?
Who else would use tools like Copilot?
@GossiTheDog @nadia_z
Sorry for that spontaneous bad habit style of in-questioning of serious studies.
But I cannot imagine that they could make studies on groups of persons, which are not biased in pro or contra LLM-habits.
@Nowhereman @GossiTheDog @nadia_z I’m a little rusty, but I could whip the shit out of an LLM at Contra.
@GossiTheDog
Isn't that an intentional side effect? 😎
@GossiTheDog but Microsoft continues to push copilot regardless.
@GossiTheDog that is exactly intentional als installing wannabe-"#AI" as 'new clergy' / 'oracles' is desireable for #Cyberfacists!
@GossiTheDog overdependence on search engines has similar results for the same reasons

@sawaba @GossiTheDog To be fair, the same can be said of any tool like GPS making you bad at orienting yourself or Stack overflow making you lazy.

Being uncomfortable and struggling is part of problem solving and one should put themselves in that position to stay fresh.

@dufresnetech @sawaba @GossiTheDog

So true. I used a GPS for months on my commute to school only to not remember how to get there.

The moment I printed out instructions, could I begin to recall my trip

I think mapquest is making a comeback :P

@sawaba @GossiTheDog Strong disagree. Use of search engines is research as it's supposed to work, and builds cognitive capability. It might reduce rote memorization capability, on the basis that you're conditioned to know you can find basic facts when needed.

@dalias @sawaba @GossiTheDog
I suspect this depends on whether one uses a search engine to look up facts and forget them until the next time you need a search engine to “remember” them

Versus using a search engine as if you were using a really big card catalogue to look up resources you want to add to your research

Like I have a huge database on my computer of files and PDFs that I looked up using search engines

I’m not sure how else I would have acquired them

That’s how I wrote my book

@dalias @sawaba @GossiTheDog
All that computer stuff may not have expanded my memory

If I had huge paper filing cabinets it would last longer if the digital apocalypse happens

But I assume people on LLMs not just for searching but for writing papers and such

That seems like it would be terrible for learning

If I was a rich parent at a #AI company, I would send my kids to a school that didn’t allow phones or computers with this stuff on it

@GossiTheDog While doubling down on Copilot...
@GossiTheDog When I see headlines like this I always ask myself if they have reversed cause and effect.
@callionica @GossiTheDog Surely there's an effect in that direction too, and maybe it's revealing bad underlying tendencies in people who formerly appeared otherwise smart. But based on what's happened to folks who embrace it and become deluded by it, I'm pretty sure the forward direction is happening too.