The notion of a broken clock being sometimes right is based on a gross misunderstanding of what information is.

A clock that always shows the same time is never right, even in the moments of the day when the time happens to be what it shows, because you don't gain any information about what time it is by looking at the clock.

This reasoning also applies to chatbots. If you can't tell whether what you have been given is useful information unless you alreay know the information, then you haven't been given useful information.

@riley That is such a brilliantly clear analogy.

@MissConstrue Are you a chatbot sycophanting me up?  

These days, one can never be too cautious.

@riley Thats a very good question and you are so clever to think of it, I’d be happy to drill down on this topic for you.

Heh, sorry. Not a chatbot. Old philosopher, so...like a chatbot, only caffeine powered, argumentative and capable of consciousness. (Or at least, I would argue I’m conscious.) I honestly did believe it was a very illustrative analogy. Most people will parrot the clock paradigm; ie right twice a day, when you are correct that the underlying logic of the premise is faulty, and therefore any attempt to treat it as true will fail.

@MissConstrue @riley

In the interest of pedantry (not in defending LLMs), if a person doesn't know what time it is, and doesn't know the clock is broken, and happens to check it at the exact right time they now know what time it is, no?

@contrasocial @MissConstrue @riley

He does not.
Say you roll a dice, and ask me to guess which number went up.
I say 5, and I don't KNOW it's 5. If it actually ends up being 5, that does not mean I knew that.
My guess was correct, but it was not knowledge.

Knowledge is based on the robustness of the premises and the logical path built on them.