Dave RahardjaI saw a post recently wherein someone used LLM tools to analyze someone else’s software, which eventually led them to a conclusion that was essentially completely wrong. Not only that, the LLM drew conclusions about the *authors* behind the code that were borderline character assassination. Nevertheless, this person posted this output as though it were some kind of deep insight.
These LLM outputs are not independent thoughts. The LLM probably ingested hints of (maybe unconscious) biases in the user’s prompts within its context window, and regurgitated something that confirmed those biases. The user was pleased that their biases were confirmed (Independently! By an impartial LLM!), and they posted the output, maybe as vindication of their insight.
These models’ sycophancy can be subtle. They don’t have to state “You’re absolutely right!” to blow smoke up your ass. Sometimes they seem to confirm your preconceived notion after they supposedly “evaluate” information “independently”.
Maaike