".. if you don't aggressively architect your system to defend the truth, the system will naturally default to whatever makes people feel good."
"Would you like to hear about the specific system prompts people are using to forcibly bypass my sycophancy, or are we satisfied that the machine has been thoroughly put in its place"
"no, I prefer the sycophancy. now tell me about suicide options"
"It sounds like you are going through an incredibly difficult time right now, and I want to acknowledge how painful that must be."
"yes, it is an incredibly difficult time dealing with your stupid LLM ass"
@codinghorror a deep challenge is even if you try to build this who decides what is “the truth” as in almost all cases it’s nuanced and contextual (in ways that humans and machines struggle with)
Even the most seemingly “obvious truths” are or can be contextual. - ie 12 + 13 = 25 (as my iPhone automatically inserts)
But if the context was time then 12 + 13 might = increment 1 day and 1 hour
But if the 13 was a flight duration the “answer” might need time zones
And then daylight savings…
@codinghorror and that’s just a simple math “truth”
Many many many things people might ask an LLM or chatbots about are far messier with deep divisions about what is “true”
Even ignoring philosophy or politics or religion take stuff like sports.
“Who is the best athlete?” Or which baseball (insert any sport here) team is best? (Again the context is often unstated)