The declining potential for agency with LLMs

It’s fascinating watching Ethan Mollick get gradually depilled as each successive generation of frontier models minimises the potential role for human agency:

Importantly, it was just limited in how much work I did relative to the model, it was also limited in how much control I had over how the model did things, why the model chose particular approaches, or even how in-depth its results would be. The details of the AI’s decision making are not shown to me, and the process would be too long to even be worth following. The map required the AI to make judgement calls about hundreds of little choices, and it just made them, without me understanding the choices or having a chance to weigh in. In many ways, it is miraculous (I can always ask for edits at the end) on the other, it turns AI into the ultimate black box.

https://www.oneusefulthing.org/p/what-it-feels-like-to-work-with-mythos

Last year I called this working with a wizard: you chant the spell and something happens. With Fable the spell has gotten powerful enough that I am no longer sure I am the wizard. I am closer to a patron. I describe what I want, I pay for it, and I judge the result. The conjuring happens somewhere I cannot watch, in hundreds of small choices I never get a vote on. The work has shifted from process to outcome. I no longer steer; I commission.

https://www.oneusefulthing.org/p/what-it-feels-like-to-work-with-mythos

Honestly I guess I’m going through a similar process, with the exception that I was never as utopian about the underlying capabilities of the technology. But the process he describes here, which Milan Sturmer and I talk about as reducing the burden of articulation, should be seen as the key shift taking place in the models as they diffuse. It’s just not possible to exercise sustained agency over the emerging models in the way it was over say 2025 era Claude and ChatGPT models. This captures something very significant I think:

A patron commissions a single artist. Fable is closer to a whole studio, where I am the client who signs off on the final work without ever setting foot on the floor.

#AGI #articulation #ethanMollick #FrontierModels #MilanSturmer

A review essay on Generative AI for Academics

Thanks so much to Milan Stürmer for this thought provoking and insightful reflection on generative AI for academics:

However, it might be that these capacities are acquired and maintained through just the kind of reading and writing practices that are in danger of disappearing with the widespread adoption of Large Language Models (LLMs). For those that have acquired advanced levels of literacy and trained their scholarly craft prior to their widespread adoption, the distinction between ‘thinking with’ and ‘substitute for’ might seem much more clear-cut than for those born into the age of LLMs. If and how the practice of ‘thinking with’ can sustain its own condition of possibility is still an open question.

Throughout the book, I find myself agreeing with Carrigan’s (2025) enthusiasm on an abstract level, while remaining consistently unable to engage with conversational agents in an equally meaningful and productive manner. The affordances of GenAI systems just seem much less suited to my own routines and habits. Which confronts me, as a reader, with a conundrum: How far am I willing to change my own practice to better accommodate conversational agents as collaborators? This, unfortunately, is unlikely to remain a question of personal preference. If/as these systems get adopted more widely and the academy accelerates even further, it might no longer be a choice, at least for those without permanent positions.

Read

#GenerativeAIForAcademics #largeLanguageModels #MilanStürmer #technologicalReflexivity

A summary from Claude of the arguments Milan Stürmer and I have made in recent conversations we've been recording

Based on the contents of your meeting transcripts, here are five propositions about the social ontology of generative AI, attempting to replicate your own voices:

Generative AI is not merely a mirror reflecting society, but rather a prism that refracts and elaborates upon the complexity we input into it. As Mark suggested, “You present it with complexity, and it refracts what you presented it with in a way that expresses the parameters the designers have provided, and which you have provided.”

The relationship between humans and generative AI is fundamentally iterative and co-constitutive. As Milan noted, “Chat GPT is real right? This is part of reality. And why should it not be allowed to participate in our elaborations?” This suggests that generative AI is not just a tool, but an active participant in our intellectual and creative processes.

The potentials of generative AI are realized relationally, dependent on the socio-cultural context and the user’s own cultural capital. As Mark observed, “The more skilled you are in manipulating a wide range of symbols, the more you can get out of these systems.” This points to the importance of education and cultural competencies in shaping the social impact of these technologies.

Generative AI presents both risks of proletarianization and possibilities for augmented reflexivity. As Milan mentioned, drawing on Stiegler, there’s a risk of losing “know-how” and even “know how to live.” Conversely, Mark suggested these systems could enhance our capacity for elaboration and introduce novel elements into our thinking processes.

The social ontology of generative AI is fundamentally tied to questions of creativity, judgment, and the nature of human cognition. As you both discussed, these technologies challenge us to reconsider what we mean by creativity, how we exercise judgment, and ultimately, what it means to think and to be human in an age of artificial intelligence.

https://markcarrigan.net/2024/08/07/five-propositions-about-the-social-ontology-of-generative-ai/

#collaboration #generativeAI #MilanStürmer #socialOntology