Mastodawn

Wim Godden 5d ago

Riley S. Faelan

The notion of a broken clock being sometimes right is based on a gross misunderstanding of what information is.

A clock that always shows the same time is never right, even in the moments of the day when the time happens to be what it shows, because you don't gain any information about what time it is by looking at the clock.

This reasoning also applies to chatbots. If you can't tell whether what you have been given is useful information unless you alreay know the information, then you haven't been given useful information.

Riley S. Faelan Mar 11

This confusion is also what cold reading is based on, btw. Falling for a chatbot is literally the same type of mistake as falling for a psychic telling you that somebody you used to know who had a vowel in their name died.

Uilebheist Mar 11

@riley But in my infinite knowledge, I can also add that they died on a day of the week ending with "y"!!!111

Riley S. Faelan Mar 11

@Uilebheist The German version is, they died on a masculine day. (German has one day of the week that does not end in -g, der Mittwoch, but all days of the week are masculine.)

George B Mar 11

@baldur 's LLMentalist article is something I have shared repeatedly since it first came out:

https://softwarecrisis.dev/letters/llmentalist/

The LLMentalist Effect: how chat-based Large Language Models rep…

The new era of tech seems to be built on superstitious behaviour

Out of the Software Crisis

Riley S. Faelan Mar 11

@gbargoud Thanks! I didn't know of this article.

@riley cold callers like this have always struggled in the Polish community

Riley S. Faelan Mar 12

@ASprinkleofSage That doesn't sound right. Are you telling me that Polish people don't have vowels remaining in their names by the time they die?

Now, if you were talking about Serbian people, maybe ...

@riley what i do know is y and z score very low in Polish scrabble...

MissConstrue Mar 11

@riley That is such a brilliantly clear analogy.

Riley S. Faelan Mar 11

@MissConstrue Are you a chatbot sycophanting me up?

These days, one can never be too cautious.

bdf2121cc3334b35b6ecda66e471 Mar 11

@riley @MissConstrue I am not a bot. Please don't look at my name.

MissConstrue Mar 11

@bdf2121cc3334b35b6ecda66e471 @riley
01001001 00100000 01110011 01100101 01100101 00100000 01111001 01101111 01110101

;)

Captain Button Mar 11

@riley @MissConstrue

Are you very concerned that a chatbot sycophanting you up?

Riley S. Faelan Mar 11

@cptbutton Tell me about your parent directory.

</eliza>

MissConstrue Mar 11

@riley @cptbutton I never really knew my root...

MissConstrue Mar 11

@riley Thats a very good question and you are so clever to think of it, I’d be happy to drill down on this topic for you.

Heh, sorry. Not a chatbot. Old philosopher, so...like a chatbot, only caffeine powered, argumentative and capable of consciousness. (Or at least, I would argue I’m conscious.) I honestly did believe it was a very illustrative analogy. Most people will parrot the clock paradigm; ie right twice a day, when you are correct that the underlying logic of the premise is faulty, and therefore any attempt to treat it as true will fail.

Riley S. Faelan Mar 11

@MissConstrue There's an interesting pattern to a large number of these faults, but I guess it'll be a topic for another day.

Alex@rtnVFRmedia Suffolk UK Mar 11

@riley @MissConstrue

I was thinking of some equipment I saw at a "Telekom-Museum" in Germany - it contained a clock but wasn't always powered on (or was just a display piece)

The Germans had quite sensibly put a diagonal strip of red tape (in the style of the "Universal No" symbol) across the clock face, so you knew it was *not* a timepiece to be trusted..

Riley S. Faelan Mar 11

@vfrmedia In aviation, the process is standardised by way of the INOP stickers.

contrasocial Mar 11

@MissConstrue @riley

In the interest of pedantry (not in defending LLMs), if a person doesn't know what time it is, and doesn't know the clock is broken, and happens to check it at the exact right time they now know what time it is, no?

MissConstrue Mar 11

@contrasocial @riley They may, but that still doesn’t make the clock correct. Truth can be derived from false premise, logically speaking, ie symbolic, but, and this is the important bit, it does not make the false premise true.

contrasocial Mar 11

@MissConstrue @riley

Right, the premise that the clock will be correct when looked at is false regardless of the truth of what time it is. Makes sense.

❣️u.s. pause federal tax remittances Mar 12

@contrasocial @MissConstrue @riley

so now, i would like to acquire a circular object with marks around the edge and hands which are still... and carve or paint 'ceci n'est pas une horloge' on the face... oh my i may need to sketch or paint a version of the pipe to hang nearby, maybe in portuguese.

MissConstrue Mar 12

@melioristicmarie @contrasocial @riley

I have a clock that a friend painted to match my chairs, because at the time I had chairs of many colors. Something in the mechanism stopped working years ago, and I've long since changed furniture, but I still have the clock because it's pretty and a friend made it. Randomly one of us will move the hands. It can be purple o'clock at a whim, but tread carefully indeed should the clock say green-thirty.

Michael Weiss Mar 12

@MissConstrue @contrasocial @riley this is something I harp on frequently. If you're right for the wrong reasons, it's worse than being wrong. You got there by coincidence, but now you believe you know how to get there. That makes you far more dangerous than someone who simply doesn't know.

Riley S. Faelan Mar 12

@contrasocial No. You're trying to construct a stack of an even number of errors, but there's other even numbers than zero.

@contrasocial @MissConstrue @riley

He does not.
Say you roll a dice, and ask me to guess which number went up.
I say 5, and I don't KNOW it's 5. If it actually ends up being 5, that does not mean I knew that.
My guess was correct, but it was not knowledge.

Knowledge is based on the robustness of the premises and the logical path built on them.

Joyously following up on the pedantry; the definition of knowledge I like best is _justified true belief_. This person doesn’t have justification though they think they do.

Our justification is often shakier than we’d like!

@contrasocial @MissConstrue @riley

ChookMother 🇦🇺🦘Mar 15

@contrasocial @MissConstrue @riley Yes but only in the sense that you can pick the "right" number in a lottery. A co-incidence is not knowledge.

Andii אַנדִֽי 5d ago

@MissConstrue @riley
There is a sense in which it is right bidiurnally, the issue is that without an independent corroboration, a viewer has no way to know when is one of those two times. And why would you bother?
Interesting to consider also the case of my watch which gains time over a month or so. It is never right but is more useful because it is close enough for most purposes and I can make allowance for its growing inaccuracy.

MissConstrue 5d ago

@Andii Mostly unrelated, and definitely anecdotal; I love watches. Adore them. Analog, filled with gears and things that go around, watches. I'm common, so most of my watches have been inexpensive tat, but I inherited a very good one. And I've discovered that no matter the quality of the timepiece, they will react oddly when I wear them. They will randomly speed up or go backwards. Literally backwards. I've not worn a digital one, so I can't speak to them, but something about me makes watches go haywire. Strangest damn thing.

Tobias Mock Mar 12

@riley @MissConstrue You're absolutely right!

Prof. Rachel Thorn 🍉🇺🇦🏳️‍⚧️🏳️Mar 11

@MissConstrue @riley What Miss Construe said.

@riley Hmmm. I think you got that one wrong. The point of the figure of speech is not to give credit to the clock. The point is to point out that the information is useless.

Riley S. Faelan Mar 11

@proedie No, that's not how information works. Information is about reducing your uncertainty space. Every time you can exclude half of the uncertainty space, you will have gained one bit of information. If you exclude less than half of the uncertainty space, you will have gained less than a bit of information. Just ask Claude[1].

Looking at broken clock[2] does not reduce your uncertainty space at all, therefore you gain zero bits of information. The classic formula Claude Shannon is famous for involves dividing the volume of the uncertainty space after gaining information with the volume of the uncertainty space before gaining information, and then taking a base-2 logarithm of the ratio and negating it. If you don't care a minus one bit about negative amounts of data, you can turn the ratio on its top; then, negation won't be necessary. But there's didactic reasons for presenting it in the classic way.

[1] Claude Shannon, an overall smart human and a measurer of the enthropy of information. Who were you thinking about?
[2] Well, there's the minor issue of knowing that the clock is broken, lest you erroneously throw out parts of your uncertainty space that might actually be valid. But the problem of information-resembling text is also an issue that applies to chatbots.

@riley That’s the point. You got information theory right. You just misunderstood the expression with the clock.

When I say: ‘My AI gave me a correct answer once’, you can reply: ‘Sure, even a broken clock is correct twice a day.’ Thus stressing that coincidental correctness is worthless.

Renke Meuwese Mar 11

@proedie @riley exactly. This is not countering the proverb, this *is* the proverb.

jonoleth Mar 11

@proedie @riley given a cursory googling and this reddit poll, it doesn't seem like the meaning is that clear to the average person

https://www.reddit.com/r/polls/comments/1brhoj4/how_do_you_interpret_the_saying_a_broken_clock_is/

jonoleth Mar 11

@proedie @riley after obsessing a little over getting to the bottom of this, the answer seems to be that the historical origin (from 1711) is akin to "If you stop chasing trends you will sometimes be fashionable", which is more in line with riley's definition in the OP. The other "official" definitions I've found seem to follow this as well.

The definition that "coincidental correctness is worthless" seems to be a personal (though common) interpretation.

@jonoleth @riley Oh, wow. Could you share the source with us?

Riley S. Faelan Mar 12

@proedie Sounds like something that friends of Beau Brummell might have noticed, but he lived too late for that.

dragonfrog Mar 12

@jonoleth @proedie @riley I always thought of the saying in re those people (often conspiracy theorists) who are singularly obsessed with one thing and always turn to that thing as their explanation for the whole world around them. Occasionally their obsession actually will be the correct answer but it's not like you gain any insight from listening to them.

It's not an acknowledgement of their insight, but a wry and half mocking acknowledgement that today, for once, they get to be right.

SarahATX Mar 11

@riley @proedie thanks for sharing... I think this is going to send me on a little research rabbit hole this afternoon!

Kaitlyn~Ethylia Mar 11

@riley @proedie the expression is using a coloquial definition of information, not a rigorous one. A clock "gives us information" when we look at it: whether it's right or not, by virtue of being a clock it is asserting that "this is the current time"

Kaitlyn~Ethylia Mar 11

@riley @proedie don't get me wrong, I agree with your point and the analogy makes sense. I just disagree with your disagreement with the expression, it's a moot point

neocat_flag_genderfluid

@riley this is a good point, but it should also be noted that some types of information can be difficult to obtain but easy to verify.

Riley S. Faelan Mar 11

@evening That is true.

Lars Marowsky-Brée 😷Mar 11

@riley This misjudges how and why stochastic algorithms work.

(I am not saying that there is no AI hype, nor that they're ethical.)

Riley S. Faelan Mar 11

@larsmb I'm not entirely sure I understand your point (I might if you fleshed it out some more), but I suspect that a relevant counterpoint you might not have properly considered is, the uncertainty space doesn't have to be flat. It can have an extra axis of plausibility, allowing for fuzzy exclusion of points on it, not just a black-and-white excluded/included binary.

Lars Marowsky-Brée 😷Mar 11

@riley I blame my undercaffeination, you *did* imclude that via the "if you can't tell" part.

My apologies for a redundant reply.

GreenSkyOverMe (Monika)Mar 11

@riley Yes, this!

@riley I have to say the analogy is so on point, it autistically satisfied my brain (and my left foot (I don't know why)).

zombiecide　馬雄 Mar 11

@riley Long before the advent of chatbots I found myself musing about the role "trust" plays when receiving information, based on personal interaction in primary groups, based on roles in secondary groups, based on rules and regulation in tertiary groups, and how internet interaction was a new type that could be any of those or different, but many people tend to use primary group patterns and assumption of familiarity, like conning yourself
chatbots do this as a service?

Riley S. Faelan Mar 11

@zombiecide Well, what did your pondering found out about the combination of — it's purely hypothetical, with absolutely no connection to anything in real world because nobody would ever do something silly like that — the possibility of applying trust heuristics to a bunch of anonymous people writing in a wikiwiki about the sort of stuff that one might look up in an encyclopædia?

zombiecide　馬雄 Mar 11

@riley funnily enough, in the time between those first, completely unquantifiable, musings and now many people's trust in a wikiwiki encyclopædia increased a lot, partly maybe because of the transparency of its process, and to another part, maybe due to familiarity? with discussions in the media, at school and workplaces about when and how and what for to trust

TrimTab 🇺🇦Mar 11

@riley
I love this post, very thought provoking.

As a native English speaker I have never once conceived of the idiom about broken clocks meaning what you say though, regarding gaining knowledge.

In my experience it is used to mean someone/thing is sometimes right, but not from any action they took, rather through luck, error, whatever. They are the broken clock.

I love your take though and the point as a whole.