My students are often surprised to learn that LLMs aren’t answering their questions. Rather, an LLM answers the question “what would a reply to this look like?” It’s one of the first things I explain in the “Should I use LLMs?” portion of my syllabus.
@mcnees While I agree thats important to keep in mind, what is not clear is the degree to which people also answer questions that way. LLMs are definitely not what we believe intelligence to be, but could it be that that belief is incorrect?
@scottfweintraub @mcnees LLMs have no investment in their answer. If you tell them they're wrong, they'll just reroll the dice to try to make you happy. A person wont do that if they know they're right.