Mastodawn

David Chisnall (*Now with 50% more sarcasm!*)Jan 1

A lot of the current hype around LLMs revolves around one core idea, which I blame on Star Trek:

Wouldn't it be cool if we could use natural language to control things?

The problem is that this is, at the fundamental level, a terrible idea.

There's a reason that mathematics doesn't use English. There's a reason that every professional field comes with its own flavour of jargon. There's a reason that contracts are written in legalese, not plain natural language. Natural language is really bad at being unambiguous.

When I was a small child, I thought that a mature civilisation would evolve two languages. A language of poetry, that was rich in metaphor and delighted in ambiguity, and a language of science that required more detail and actively avoided ambiguity. The latter would have no homophones, no homonyms, unambiguous grammar, and so on.

Programming languages, including the ad-hoc programming languages that we refer to as 'user interfaces' are all attempts to build languages like the latter. They allow the user to unambiguously express intent so that it can be carried out. Natural languages are not designed and end up being examples of the former.

When I interact with a tool, I want it to do what I tell it. If I am willing to restrict my use of natural language to a clear and unambiguous subset, I have defined a language that is easy for deterministic parsers to understand with a fraction of the energy requirement of a language model. If I am not, then I am expressing myself ambiguously and no amount of processing can possibly remove the ambiguity that is intrinsic in the source, except a complete, fully synchronised, model of my own mind that knows what I meant (and not what some other person saying the same thing at the same time might have meant).

The hard part of programming is not writing things in some language's syntax, it's expressing the problem in a way that lacks ambiguity. LLMs don't help here, they pick an arbitrary, nondeterministic, option for the ambiguous cases. In C, compilers do this for undefined behaviour and it is widely regarded as a disaster. LLMs are built entirely out of undefined behaviour.

There are use cases where getting it wrong is fine. Choosing a radio station or album to listen to while driving, for example. It is far better to sometimes listen to the wrong thing than to take your attention away from the road and interact with a richer UI for ten seconds. In situations where your hands are unavailable (for example, controlling non-critical equipment while performing surgery, or cooking), a natural-language interface is better than no interface. It's rarely, if ever, the best.

Erik Jonker Jan 1

@david_chisnall ...that's also one of it's strengths, language is a completely different "beast" then math. Comparing it is useless. Language fulfills different functions then math. But just as important for human beings.

@ErikJonker @david_chisnall

That's the point. Giving directions to a machine requires math. Using the term "language" to refer to machine instructions has led people down the wrong path over and over again and led to monstrosities like COBOL and Perl and "Congratulations, you have decided to clean the elevator!"

Erik Jonker Jan 1

@resuna @david_chisnall cobol is not a monstrosity as a programming language, it’s ofcourse legacy

Erik Jonker Jan 1

@resuna @david_chisnall …a natural language interface for a computer can have enormous benefits, a good example is an educational context, you can interact, ask question’s etc in a way not possible before

@ErikJonker @david_chisnall

Even communicating with other humans in natural language leads to confusion, and humans are much much better at dealing with ambiguity than any computer.

Erik Jonker Jan 1

@resuna @david_chisnall ofcourse but current AI models can provide a level of education that scales easily, it will supplement humans in their roles and sometimes replace them. Current models can perfectly help students with high school math, even with some ambiguity

@ErikJonker @david_chisnall

The software that people refer to as "AI" is nothing more than a parody generator, and is really really bad at dealing with ambiguity. It's a joke. If you actually think that it is capable of understanding, then it has been gaslighting you.

Erik Jonker Jan 1

@resuna @david_chisnall I actually know how these models work, it's not about intelligence and understanding, they are just tools but very good ones in my own experience

Erik Jonker Jan 1

@resuna @david_chisnall ...if you tried GPT-4o or a tool like NotebookLM then you know they are more then parody generators, it doesn't help denying the capabilities of these technologies especially because there are real risks/dangers with regard to their use

Cluster Fcku Jan 1

@ErikJonker @resuna @david_chisnall now take your comments, and substitute it like this: "I find English and German very useful for work. It doesn't help denying the capabilities of *natural languages* especially because there are real risks/dangers with regard to their use". At times language appears as outer thought, but do not use it is as decisive thought. As a centralized source for inquiry and digestion, LLMs are far more dangerously illusive than the natural languages by billions.

@ErikJonker @david_chisnall

They purely operate on text patterns, they do not reason, they do not build models, they just glue tokens together. There is nothing in their design to do any more than that. This is an inherent feature of any example of this class of programs.

naught101 Jan 1

@ErikJonker @resuna @david_chisnall

Huh? Perfectly?

There have been multiple instances of people showing LLMs getting answers wrong to the most basic arithmetic problems. That's not a bug, it's an inherent feature of the model, which draws meaning from language only and has no concept of maths.

That incorrectness can only get more likely as math problems get more complex. And the more complex it gets, the harder it is for humans to detect the errors.

How is that perfect for education?

Erik Jonker Jan 1

@naught101 @resuna @david_chisnall as a support tooll during homework, where it can give additional explanation, I see a bright future for the current best models (for highschool level assignments) , for text based tasks they are even better (not strange for LLMs) . Ofcourse people have to learn to check and not fully trust, at the same time there is a lot of added value. It's my personal/micro observation but i see it confirmed in various papers

@ErikJonker @naught101 @resuna @david_chisnall

Of course people have to learn to check and not fully trust,

This is what makes them particularly ill-suited for educational tasks. A large part of education on a subject is developing the ability to check, to have an intuition for what is plausible.

Erik Jonker Jan 1

@RAOF @naught101 @resuna @david_chisnall true, but you can adapt and fine tune models for that purpose

@ErikJonker @naught101 @resuna @david_chisnall can you? How? Is there an example of this that you have in mind, or is this more a “surely things will improve” belief?

What is the mechanism that bridges “output the token statistically most-likely to follow the preceding tokens” and “output the answer to the student's question”?

@ErikJonker @naught101 @resuna @david_chisnall also, isn't the task you're suggesting is possible just equivalent to “make an LLM you don't need to check the results of”?

Violet Madder Jan 2

@ErikJonker @naught101 @resuna @david_chisnall

The support students need for their work, is A HUMAN BEING WHO IS GOOD AT UNDERSTANDING AND EXPLAINING THINGS.

A good teacher/tutor/sibling/etc can break down an explanation and present it in different ways tailored to the student's understanding. They can look at a student's work and even if it's incorrect, see what the student's train of thought was and understand what they were trying to do.

Our society already drastically undervalues that crucial, mind-accelerating work-- arguably the most important of all human endeavors, as everything else relies on it.

Glorified stochastic parrots spewing botshit are no damned substitute.

@ErikJonker @david_chisnall

My first job was programming in COBOL before it was legacy. It is terrible. It always was terrible. It's not a natural language, it's not ambiguous, but trying to make it look like a natural language was an unmitigated disaster at every level. The same is true of Perl's "linguistic" design. Even just pretending to be a natural language spawns monstrosities.

Edit: see also Applescript.

Erik Jonker Jan 1

@resuna @david_chisnall it is extremely stable and durable for sure, ask any financial institution 😃

jhannafin Jan 1

@ErikJonker @resuna @david_chisnall OK, but that's not because of COBOL. You could write something durable and stable in any programming language. Financial software is written in COBOL because that was the language of the mainframe at the time. The fact that it's still largely in COBOL is because it's expensive to rewrite, the returns on a rewrite are hard to quantify, and the risks are huge.

@ErikJonker @david_chisnall

Have you ever written any code in COBOL? Everything in COBOL takes longer to write, the fundamental operations are simplistic and verbose, the program structure is stilted and restrictive, the way you define data structures is horribly antiquated, and a huge number of the problems that make writing COBOL so slow and painful are due to its mistaken "language like" design.