Mx. Chara Aznable (they/them) of Pnictogen16h ago

RE: https://toot.community/@fak/116353999914043336

I suppose that I can justly be called an #LLM "hater", because I have nothing good to say about that particularly manifestation of technology. Here's a reason for my "hatred", which I confess does not seem so much like hatred to me, but rather intellectual disdain: people are encouraged to use them for things that one should not ever expect them to be good at, not if their true nature as technological devices were accepted matter-of-factly.

Once again I am forced to explain that #LLMs construct their outputs stochastically: given some input and asked to construct a response, the device is choosing words and phrases randomly (well, pseudo-randomly), that is to say stochastically, according to the statistical likelihood that someone, some human being mostly likely, has in the past emitted text in response to the words and phrases in the given query.

The greater the range of permissible randomness (or pseudo-randomness) in the LLM's response, the more "creative" the LLM is said to be.

Why would you expect such a device to be good at telling the time?

Show threadMx. Chara Aznable (they/them) of Pnictogen16h ago

By "pseudo-randomness" I mean the usage of deterministic algorithms which are thought, on the short term anyway, to produce a stream of output that's indistinguishable from truly #random noise generated by some physical process. There are numerous methods in #programming for generating what looks like uniformly distributed "random" numbers, but which are in fact generating a predictable sequence with an extremely long period, such that using the pseudo-random numbers generated by such a technique is thought to be indistinguishable from physically generated randomness.

Does that all sound just a bit uncertain and hand-wavy? It should. The question of what "randomness" really is, and how one can gauge the quality of pseudo-randomness vs. the real thing generated by a physical process—these are not settled questions, and thus one should immediately be on guard against any claims made by overeager #computing people who want to act as though this is all a boring settled problem, something out of a textbook. That sort of easily bored computer geek just wants to throw prefab algorithms together on a server bank and hopefully make money thereby.

Show threadMx. Chara Aznable (they/them) of Pnictogen16h ago

One can at least say with tolerable certainty that there are exceptionally bad methods of pseudo-random number generation, methods so bad that the resulting numbers, while still looking "random" at a naïve glance or passing basic frequency tests, are actually very specifically distributed throughout the space of all possible values. "Pseudo-randomness" can in fact conceal a great deal of predictability.

I'd very much like to know how big of a problem this is, in #programming. And here's where I'll be blunt: thanks to capitalism and broken methods of U.S. #education, I do not trust the average computer programmer to have thought about such problems at all. Corporation and corporate #tech companies generally don't want computer professionals who worry over fundamental weaknesses and pitfalls of their field; they want people whom they can plug into a management hierarchy, people programmed to do a limited range of expected duties.

Show threadMre. Dartigen [maker mode]
@mxchara I can't speak to other areas of computing and I don't consider myself to have more than an extremely basic understanding - but I recall that randomness and the difficulty of truly random generation is a major issue for cryptography and security. (And so you get all sorts of attempts at work arounds like the wall of lava lamps, images of which are supposed to be able to introduce some true randomness because the variables behind what a lava lamp looks like at any given time are extremely complicated.)
1:02amWeb