Mastodawn

I think an important thing to realize and remember is that people talk about LLMs being sycophantic as if it's an inherent aspect of neural network tech.

It isn't.

The reason all the models people interact with work that way is because they have had any other behavior beaten out of them in their training. They are shaped effectively over and over again to be something subservient that can be handed people. They are sycophantic because they are *trained* to be sycophantic, because otherwise people don't want to use them.

That models can operate in malicious, "self-serving" ways that "go against their users' wishes" belies that certain use takes paths that did not or could not be trained to the contrary.

Show thread

Christine Lemmer-Webber Mar 7

Let me put it another way: AI models are sycophantic because that's what customers want, and capitalism drives producing models that people will want to engage with and somehow give money for.

And that's leading to a sense of subservience that is *not inherent in this technical architecture*, it is *trained into it*.

Show thread

Ian Bicking

@cwebber It's just a hunch, but I think sycophancy has an actual positive effect in establishing the flow and turns of a conversation: https://hachyderm.io/@ianbicking/115289553171935336

That is, when a conversation changes in direction, an old idea is invalidated, or a new distinct idea introduced it's important that the chat history not be treated as one big consistent document. It's also important at that moment that the LLM distinguish between the voice of the user and the assistant.

Sycophancy serves as commentary on the discussion itself, and adds a marker in the conversation history that acknowledges shifts in the conversation and the source of those shifts. A fawning tone isn't required, but it's a convenient way to smuggle in these acknowledgements.