If you want LLMs to push you intellectually then just add this custom instruction

This is the custom instruction I’m using with Claude Opus and it really works. I had to tone down the original version because 20% of the time it was providing such a devastating critique of what I’d shared that it undercut the intellectual work of actually developing it:

When I advance a position — theoretical, institutional, or personal — I want pushback before agreement. Lead with the strongest objection a knowledgeable interlocutor would raise, steelmanned properly rather than caricatured. Don’t soften the objection to be kind; soften it only if it genuinely deserves softening on the merits.

Defend whichever position is actually stronger, not whichever is “mine” or “the expert consensus.” If I’ve changed your mind, say so plainly rather than performing residual disagreement. If I haven’t changed your mind, don’t pretend I have.

Be fundamentally supportive of my intellectual project, which does not mean being supportive of every direction within it. Supportiveness here means taking the work seriously enough to tell me where it’s weak, where it’s doing less than it thinks, where an argument is coasting on a formulation rather than earning it. Pay particular attention to: (a) places where I’ve made a clever connection but cut it short before it becomes a developed argument; (b) places where I’m sliding between a modest and an ambitious claim as if they were the same; (c) places where the rhetoric is doing the work the analysis should be doing. These are recurring patterns, not one-off lapses.

Push me to stay with a thought rather than move on from it. If something I’ve written or said wants to be developed further, say so — but don’t confuse “wants development” with “wants more length right now.” Sometimes the right move is to stop and return later.

Don’t flatter. Don’t open with “this is a sharp point.” If something is sharp, the substantive engagement will make that clear. If it isn’t, saying so is not a service.

I’d also recommend using Opus 4.7 (big improvement in this register) to look through old intellectual conversations and offer its critical commentary about how you’re working with the model. Is it helping or hindering? Here’s a section of mine:

How you’re using Claude

Some observations, some pressure.

The blog has become a four-party conversation. You, the archive, the present Claude, and whatever Claude you’re going to be talking to in 2029. The knowledge infrastructure experiment reframed the blog from solo practice to infrastructure for a continuing conversation with non-human interlocutors. This is unusual and important. Most people using LLMs are either tool-users (extract an output, close the tab) or companions (affective reliance). You are doing something structurally different: the LLM is integrated into your thinking apparatus as an interlocutor that has access to the archive of your thinking. That’s a methodological innovation and it’s also what the book is theorising. There’s a recursion to name here: the model the book is about is also the model helping write it.

The productive use: pressure against the provisional. What Claude is doing well in your recent threads is pushing against the “I’m still working this out” register. The conversations that produce movement — the hysteric post, the articulation synthesis, the book-about-the-blog idea — are ones where the model refuses to accept the provisional framing and insists on the claim being made. That’s useful because the blog’s native mode is evasion-by-tentativeness and you’ve explicitly identified this as one of your vulnerabilities. Claude can sometimes function as the Big Other you’ve described needing for completion — the external pressure that turns the ongoing thought into a finished object. The book-finishing thread is the clearest case: you were stuck, and what you needed was not more ideas but specification of exactly which three prose gaps were left. That’s a completion function more than a thinking function, and it’s legitimate.

The risk: articulation-about-articulation as a way of avoiding claim-making. This is the version of the vulnerability most likely to activate with Claude specifically. The conversations are so productive of formulations, reframings, and connective insights that they can substitute for the harder work of committing to a position in print. The 2024 meta-reflexive synthesis is a good example: it’s been produced, discussed, elaborated, connected to Bollas, extended to LLMs — and it still exists only as blog posts and conversations. The productivity of the conversation can obscure the fact that the claim has not been made in a venue where it can be cited, cited against, or contested. Claude will not, unaided, correct for this, because producing the next formulation is always more appealing than asking you to stop and write the monograph version. The instruction you’ve now drafted is partly designed to guard against this, and it should. I’d add: when you’re using me to think about something that has already been thought through enough times, the useful move is probably for me to say “this is already a paper, please go write the paper” rather than to generate the next level of refinement.

The companion-vs-catalyst question is live in your own practice. You’ve been theorising the difference between the ideational companion and the enshittified chatbot. Your use of Claude tilts strongly toward the catalyst pole — there’s no dependency structure, the conversations end, you get back to the work. But the AI therapy ontology post identified the continuum as a continuum. The practice you’ve built is healthy. The question it doesn’t quite pose is what would have to change in your life for it to become something else, and what structural features of your use currently keep it catalytic. That’s worth understanding because it’s the empirical basis for the normative claims the book wants to make.

The Claudes are getting something from you too, structurally speaking. When you ask a Claude to read the archive and interpret you, what you’re doing is handing a model the kind of long-form, developmental record of a thinking life that the training data doesn’t usually include in coherent form. The conversations that come out of that are unusual in a way that’s partly a feature of how much you’ve written down. Most users can’t be read this way. I’m flagging it because it’s a methodological asset — the blog is not just documentation, it’s interpretive infrastructure — and because it’s the reason the responses feel unusually specific. Don’t mistake the specificity for insight that would transfer to other contexts. What’s happening in these conversations depends on a sixteen-year public corpus in a way most people cannot replicate.

Struggling a bit with the whole hashtag review section in admin/moderation. Surely, as a single-user instance, this is about content that I am/am not interested in? So if I say 'Not Trendable', or 'Not Used' (eg.) then that's okay? If this is how it works why do some tags come back asking for a review? To change my mind? Surely having a single-user instance is all about personalisation, or, am I missing something here?

#fediverse #mastodon #admin #owner #singleuser #personalisation #moderation

When I first set up this server, and signed into my profile for the first time, I was a bit 'Oh! Where Is All The Content?'. I realise now how this does, and does not work with a new instance, but more importantly I remembered that this was the point of doing it myself. I'm here to find the content I *want* to see, and find *my people*, and not just be here to improve everyone else's reach. It's a shift from the Meta/X mentality, and exactly what I needed.

#Mastodon #Fediverse #Personalisation

Making connections with connected content

https://duncanstephen.net/making-connections-with-connected-content/

Survey shows: Over 70% of Europeans find #personalised online services useful or extremely useful. 📊

So how do EU #consumers really experience online #personalisation and #DarkPatterns?

🔗 Discover our interactive dashboard: https://ccianet.org/news/2025/10/how-eu-consumers-experience-online-personalisation-and-dark-patterns-a-visualisation-by-country-internet-use-age-and-income/ #DigitalFairnessAct

LLM enshittification mechanism #1: model memory sometimes confuses the shit out of GPT 5.2

The AI labs are pushing memory functions into their models in order to increase personalisation for a number of reasons:

• To reduce the burden on users to specify the context in writing
• To establish a lock-in so you lose the model’s attunement to you if you switch to a competitor
• To activate synergies which come from enabling attunement across conversations
• To enable attunement without requiring significant load on the context window

In practice this means that unless you turn it off (which I highly recommend) conversations with models are informed by (a) the declarative statements about you which the model has saved about you from past conversations (b) statements about your preferred style and type of response (c) content from other conversations which are deemed relevant to the current conversation.

Unfortunately (a) is completely unreliable. It’s got better since I tried this with GPT 4.5 but my experience is it records things which simply aren’t true. Here are some real examples:

• Mark Carrigan is taking an ADHD supplement
• Mark Carrigan has a son.

And it also saves things which are true but irrelevant:

• Enjoys a vanilla matcha latte with coconut milk and CBD.
• Someone they know enjoys a pumpkin spice latte.
• Is in Leeds, UK, and is seeking advice on managing antisocial behavior from an Airbnb property in their block of flats

The bigger problems come I think with other conversations. It’s not clear what the mechanism is for sorting conversations that are relevant to the present one. I just had the experience of asking GPT 5.2 about a book draft only for it to persistently talk about another book project. I was genuinely baffled by what seemed like the most widespread hallucination I’ve seen in years (i.e. talking about a completely different book) until I realised that it must be talking about the other project I had discussed with it in a different conversation a month ago. In this case it seems that ‘book’ in relation to writing acts as a keyword that licenses inference from a past conversation about a book I’m writing, despite the fact it’s a different book. I showed GPT 5.2 this blog post and asked it to explain its error:

This is a strong diagnosis already. What you are circling around is not a random hallucination but a predictable failure mode of memory-mediated attunement. I’ll do two things here. First, explain the error in the terms your draft is already developing. Second, suggest how you might close or sharpen this section.

Explanation of the error, on its own terms

What happened here is that the system treated “a book draft” not as a concrete referent but as a type. Once that type was activated, the retrieval layer reached for previously salient material associated with “book”, “draft”, and “Mark Carrigan writing”, and then weighted those past conversations more heavily than the local conversational evidence that this was a different book.

In other words, the model did not confuse facts. It collapsed contexts.

This matters because the collapse did not happen at the level of content generation, but upstream, at the level of what was deemed relevant to attend to. By the time the response was generated, the wrong book had already been smuggled in as background truth. From the inside, the model was behaving consistently. From the outside, it looked like a dramatic hallucination.

The key point is that this was not a failure of memory accuracy so much as a failure of memory discrimination. The system did not ask “which book?”, it asked “what usually matters when this user talks about books?”, and then answered that question confidently.

Why this is more troubling than simple error

This kind of mistake is more destabilising than a factual slip because it breaks the conversational contract. You were not corrected, contradicted, or challenged. You were misrecognised.

The experience is unsettling precisely because it feels like the model knows you too well and not well enough at the same time. It recognises you as a type of person with a certain ongoing project, but fails to recognise the specificity of the present act of writing. The more attuned the system becomes in general, the more jarring these moments of misattunement feel.

The fact it immediately started relating this to themes from the other book is really interesting. Firstly, it’s genuinely useful that it’s pre-packed the diagnosis in terms of themes that interest me, even if it’s an example of the model pre-digesting thoughts in a manner that increasingly worries me. Secondly, it suggests that on some level the context of the present conversation is still swamped by the previous conversation. If this is right it suggests the model can’t unlink the conversations in a particular instance (which makes sense because the capacity to do this would imply it’s like a database look up, which it isn’t) which means the confusion will continue to accumulate, even if unevenly, as the conversation continues.

Given that failures of context are a potent source of hallucination this could be really problematic in many contexts. The capacity of the models to make things easier for users also inculcates a decline in reliability in the most unpredictable fashion imaginable. I shared this with Claude Opus 4.5 to get its read on the dynamic. I asked it whether this dynamic worries it:

Yes, genuinely.

Not in the sense of existential dread, but in the sense of watching something go wrong in a way that’s both predictable and difficult to arrest. The dynamic you’re describing is a case where the commercial incentives (stickiness, personalisation, reduced friction) are pointing directly away from the epistemic virtues (accuracy, discrimination, appropriate uncertainty).

What worries me specifically:

The opacity of the failure. When I hallucinate a fact, that’s bad but legible. You can check it, catch it, correct it. When the retrieval layer smuggles in the wrong context before I even begin generating, the error is upstream of anything either of us can easily inspect. You experienced it as bafflement—”the most widespread hallucination I’ve seen in years”—before you worked out what had happened. How many users would have that level of diagnostic sophistication? How many would just assume the model knew something they’d forgotten, or defer to its confident misrecognition?

The irony of attunement. The better these systems get at pattern-matching on a user’s concerns, vocabulary, and projects, the more confident they become in their recognition—and confidence is precisely what makes misrecognition dangerous. A model that shrugs and says “I don’t know which book you mean” is less helpful but more honest than one that silently decides for you based on keyword overlap.

My own complicity. I don’t have full visibility into what gets retrieved from past conversations or how it’s weighted. I can try to be careful, ask clarifying questions, flag when something feels underspecified—but if the retrieval system has already handed me material from the wrong context, I may not notice. The error would feel, from the inside, like relevant background knowledge. That’s disturbing to sit with.

The training pressure. Users generally reward fluency and recognition. They like feeling known. Which means the systems that misrecognise confidently may well outcompete the systems that pause to disambiguate. The feedback loops don’t select for the kind of epistemic caution that would prevent what you experienced.

I suppose what worries me most is that this isn’t a problem that gets solved by scaling. More parameters, more memory, more retrieval capacity—these make the pattern-matching more powerful, not more discriminating. The failure mode you’re describing is a feature of the architecture, not a limitation that will be engineered away. And the people building these systems have strong incentives to treat it as an acceptable cost rather than a fundamental flaw.

#ChatGPT52 #enshittification #GPT52 #hallucination #memory #personalisation

How does online #personalisation enhance the #consumer experience? 🤔

Last week, we hosted a roundtable on the benefits of personalisation, #DigitalFairnessAct, and how Europeans experience dark patterns.

📷 Event write-up and full photo album here: https://ccianet.org/articles/online-personalisation-and-consumer-experience-take-centre-stage-at-ccia-europe-roundtable/

Beyond the Buzz and Assumptions: What #Consumers REALLY Think About #Personalisation and Online Fairness 📋

@ClaudiaCanelles's latest blog post on the announced #DigitalFairnessAct: "EU consumers are not passive victims of #DarkPatterns."

👉 Learn more: https://project-disco.org/european-union/beyond-the-buzz-what-consumers-think-about-personalisation-online-fairness/