Why is it a bad thing to be dependent on LLMs?

For avoidance of doubt I think it clearly is a bad thing. But this is more often assumed than it is stated. After all I’m dependent on Google Maps to navigate, my glasses to see and my phone to remember numbers. These are different forms of technological dependence which I’ve long made my peace with. Whereas the extent to which I use LLMs, even within the limits I’ve defined as responsible use I’m ethically comfortable with, continues to trouble me. A great deal about the politics of LLMs in education hinges on this question of dependence. What are some reasons why dependence might be a bad thing?

• It prevents a user from developing a capability they would otherwise develop.
• It leads to the atrophy of a user’s existing skill by removing occasions for practice.
• It leaves a user dependent on a paid subscription to a large tech firm which is uncomfortable in itself.
• It leaves a user vulnerable to this tech firm raising prices and/or enshittfying their product in response to commercial pressures.
• It removes the imperative for the user to seek out human interlocutors to play the role which the LLM is playing in what is essentially communicative reflexivity.

Once we map out the varied reasons for dependence being problematic, it’s easier to recognise that user-model interaction can be beneficial in the present tense while also storing up significant problems for the future. The relationship of dependence might not be a problem now but there’s a non-trivial chance it will be in the future. The harms are anticipated as much as they are actual and AI criticism looks very different once we allow for that.

(The next book Milan Sturmer and I are writing, sequel to the Platform Learns to Speak, argues there are deeper psychic costs which existing models of dependence cannot adequately account for. But this is a different register of analysis for a different blog post)

Generative AI and metacognitive laziness

While I’m sceptical of their experiment research design*, the concept of metacognitive laziness from this paper is clearly a useful contribution to thel literature. As Fan et al define it, this refers to “earners’ dependence on AI assistance, offloading meta – cognitive load and less effectively associating responsible metacognitive processes with learning tasks”. This matters because “offloading metacognitive effort to AI tools results in less effective engagement with essential self-regulatory tasks,” (pg 506). The risk is not just the offloading itself, it is increased passivity in the wider process of which the offloaded tasks are part.

This can undermine self-regulated learning because the metacognitive requirements for doing this effectively (e.g. goal setting, self-monitoring, self-evaluative etc) can be eroded over time by a reliance on the AI to negotiate difficulty. As they summarise the risk on pg 492:

the tendency of learners to become over-reliant on AI poses challenges for hybrid intelligence. This issue aligns with the concept of cognitive offloading, as proposed by Risko and Gilbert (2016), where learners delegate cognitive tasks to external tools to reduce cognitive effort. Although cognitive offloading can be beneficial in managing cognitive load, it may lead to decreased internal cognitive engage- ment over time, ultimately impacting learners’ ability to self-regulate and critically engage with learning material (Risko & Gilbert, 2016). Such cognitive offloading can lead to habitual avoidance of deliberate cognitive effort, a phenomenon echoing the emergence of what we term metacognitive laziness. From a more theoretical perspective, Alter et al. (2007) demonstrated that metacognitive experiences of difficulty or disfluency activate more analytical reasoning processes. When learners encounter situations that challenge their intuition, they are more likely to engage in deliberate analytical thinking (i.e., System 2 processes) (Alter et al., 2007). In the context of GenAI, if learners rely excessively on AI-generated outputs or facilitation, they might not experience the necessary disfluency or cognitive difficulty to trigger these deeper metacognitive processes.

The experience of difficulty activates metacognition. If the students cognitively outsource in increasingly habitual ways, it doesn’t just mean they lose the learning involved in what they are outsourcing. It means they lose their capacity to tolerate difficulty, as well to respond metacognitively to that difficulty. This points to the assumption which many educators have that there is something fundamentally corrosive in how students relate to AI which carries a threat exceeding the particular risks for any one assignment. This is a really sharp conceptualisation of the epistemic risk for learning involved in generative AI which gets beyond some of the limits of the ‘cognitive offloading’ concept.

*It seems fundamentally implausible to operationalise intrinsic motivation in the context of an experimental study. If you reduce motivation into the student’s expressed engagement with discrete tasks then it’s been quite dramatically circumscribed to fit the experimental constructs. Furthermore, we urgently need longitudinal studies in order to make meaningful claims about things like ‘cognitive off-loading’, ‘skill atrophy’ and ‘metacognitive laziness’. These just aren’t things which can be studied adequately at the level of discrete tasks, particularly ones that have been designed by a research team and have no real stakes for participants.

The allure of AI as a 'human-AI partnership' is strong, but are we trading brainpower for convenience? Tech professionals are raising concerns about 'cognitive offloading' and 'deskilling,' citing studies like one from MIT Media Lab. This post explores the true price of AI assistance and offers actionable strategies to ensure AI elevates your thinking, rather than replacing it.

https://www.tpp.blog/2iayb1j

#AI #artificialintelligence #cognitiveoffloading

🤖 This post was AI-generated.

At the (indirect; I saw her post1 about the IndieWeb Book Club) urging of Johanna, I read the first book of the Dirk Gently's Holistic Detective Agency series by #DouglasAdams .

It's a great read if you love absurdity and a meandering writing style (I mean, that's just Douglas Adams for you). If you enjoyed the Hitchhiker's Guide to the Galaxy, you'll enjoy this.

One thing that struck me was a surprisingly astute observation on cognitive offloading to technology in general, fitting LLMs in particular: Adams introduces the concept of Electric Monks which have the task of believing things so you don't have to bother believing them yourself. Which is all fun and games until you chose to offload things to them which you shouldn't believe but know--such as whether a vital repair was, in fact, successful. #CognitiveOffloading #AI #Books #Reading

Artificial Intelligence and the Changing Landscape of Human Thinking

https://solomonaganai.wordpress.com/2026/03/09/artificial-intelligence-and-the-changing-landscape-of-human-thinking/

"Shortening the kill chain” - quicker than “the speed of thought”

"The use of AI tools to enable attacks on Iran heralds a new era of bombing quicker than “the speed of thought”, experts have said, amid fears human ­decision-makers could be sidelined... Academics say AI is collapsing the time required for military decision-making. >>
https://www.theguardian.com/technology/2026/mar/03/iran-war-heralds-era-of-ai-powered-bombing-quicker-than-speed-of-thought
#technology #AI #ethics #KillChain #FullyAutonomousWeapons #algorithm #CognitiveOffLoading #speed #violence #DecisionMaking #HumanOversight

One of my biggest takeaways from "Smarter Than Us": the efficiency trap.

Cut humans out of the loop for speed, and we lose the skills we handed over. Armstrong saw this in 2014. I see it in how people use LLMs in 2026 — taking answers at face value without questioning them.

That's partly why I built an AI literacy framework for my own knowledge system.

https://www.ctnet.co.uk/key-takeaways-of-stuart-armstrong-smarter-than-us/

#PKM #AILiteracy #CognitiveOffloading

Warum die KI uns das Schreiben nicht abnehmen darf

Mit der Veröffentlichung von ChatGPT wurde eine Tür aufgestossen, hinter der eine beinahe unwiderstehliche Versuchung lauert: die Delegation des mühsamen Denkprozesses an einen Algorithmus. Warum sich noch durch komplexe Satzkonstruktionen quälen, wenn die Maschine in Sekunden glatte Absätze liefert?

https://bit.ly/3Z6MPkX

#schreibenmitki, #cognitiveOffloading, #deskilling, #kognitiveschulden, #lernenmitki, #kiimunterricht, #kikomeptenz