David Chisnall (*Now with 50% more sarcasm!*)Jan 1

A lot of the current hype around LLMs revolves around one core idea, which I blame on Star Trek:

Wouldn't it be cool if we could use natural language to control things?

The problem is that this is, at the fundamental level, a terrible idea.

There's a reason that mathematics doesn't use English. There's a reason that every professional field comes with its own flavour of jargon. There's a reason that contracts are written in legalese, not plain natural language. Natural language is really bad at being unambiguous.

When I was a small child, I thought that a mature civilisation would evolve two languages. A language of poetry, that was rich in metaphor and delighted in ambiguity, and a language of science that required more detail and actively avoided ambiguity. The latter would have no homophones, no homonyms, unambiguous grammar, and so on.

Programming languages, including the ad-hoc programming languages that we refer to as 'user interfaces' are all attempts to build languages like the latter. They allow the user to unambiguously express intent so that it can be carried out. Natural languages are not designed and end up being examples of the former.

When I interact with a tool, I want it to do what I tell it. If I am willing to restrict my use of natural language to a clear and unambiguous subset, I have defined a language that is easy for deterministic parsers to understand with a fraction of the energy requirement of a language model. If I am not, then I am expressing myself ambiguously and no amount of processing can possibly remove the ambiguity that is intrinsic in the source, except a complete, fully synchronised, model of my own mind that knows what I meant (and not what some other person saying the same thing at the same time might have meant).

The hard part of programming is not writing things in some language's syntax, it's expressing the problem in a way that lacks ambiguity. LLMs don't help here, they pick an arbitrary, nondeterministic, option for the ambiguous cases. In C, compilers do this for undefined behaviour and it is widely regarded as a disaster. LLMs are built entirely out of undefined behaviour.

There are use cases where getting it wrong is fine. Choosing a radio station or album to listen to while driving, for example. It is far better to sometimes listen to the wrong thing than to take your attention away from the road and interact with a richer UI for ten seconds. In situations where your hands are unavailable (for example, controlling non-critical equipment while performing surgery, or cooking), a natural-language interface is better than no interface. It's rarely, if ever, the best.

Show threadErik JonkerJan 1
@david_chisnall ...that's also one of it's strengths, language is a completely different "beast" then math. Comparing it is useless. Language fulfills different functions then math. But just as important for human beings.
Show threadResunaJan 1

@ErikJonker @david_chisnall

That's the point. Giving directions to a machine requires math. Using the term "language" to refer to machine instructions has led people down the wrong path over and over again and led to monstrosities like COBOL and Perl and "Congratulations, you have decided to clean the elevator!"

Show threadErik JonkerJan 1
@resuna @david_chisnall cobol is not a monstrosity as a programming language, it’s ofcourse legacy
Show threadErik JonkerJan 1
@resuna @david_chisnall …a natural language interface for a computer can have enormous benefits, a good example is an educational context, you can interact, ask question’s etc in a way not possible before
Show threadResunaJan 1

@ErikJonker @david_chisnall

Even communicating with other humans in natural language leads to confusion, and humans are much much better at dealing with ambiguity than any computer.

Show threadErik JonkerJan 1
@resuna @david_chisnall ofcourse but current AI models can provide a level of education that scales easily, it will supplement humans in their roles and sometimes replace them. Current models can perfectly help students with high school math, even with some ambiguity
Show threadnaught101

@ErikJonker @resuna @david_chisnall

Huh? Perfectly?

There have been multiple instances of people showing LLMs getting answers wrong to the most basic arithmetic problems. That's not a bug, it's an inherent feature of the model, which draws meaning from language only and has no concept of maths.

That incorrectness can only get more likely as math problems get more complex. And the more complex it gets, the harder it is for humans to detect the errors.

How is that perfect for education?

Jan 1 at 9:24pmWeb
Show threadErik JonkerJan 1
@naught101 @resuna @david_chisnall as a support tooll during homework, where it can give additional explanation, I see a bright future for the current best models (for highschool level assignments) , for text based tasks they are even better (not strange for LLMs) . Ofcourse people have to learn to check and not fully trust, at the same time there is a lot of added value. It's my personal/micro observation but i see it confirmed in various papers
Show threadRAOFJan 1

@ErikJonker @naught101 @resuna @david_chisnall

Of course people have to learn to check and not fully trust,

This is what makes them particularly ill-suited for educational tasks. A large part of education on a subject is developing the ability to check, to have an intuition for what is plausible.

Show threadErik JonkerJan 1
@RAOF @naught101 @resuna @david_chisnall true, but you can adapt and fine tune models for that purpose
Show threadRAOFJan 1

@ErikJonker @naught101 @resuna @david_chisnall can you? How? Is there an example of this that you have in mind, or is this more a “surely things will improve” belief?

What is the mechanism that bridges “output the token statistically most-likely to follow the preceding tokens” and “output the answer to the student's question”?

Show threadRAOFJan 1
@ErikJonker @naught101 @resuna @david_chisnall also, isn't the task you're suggesting is possible just equivalent to “make an LLM you don't need to check the results of”?
Show threadViolet MadderJan 2

@ErikJonker @naught101 @resuna @david_chisnall

The support students need for their work, is A HUMAN BEING WHO IS GOOD AT UNDERSTANDING AND EXPLAINING THINGS.

A good teacher/tutor/sibling/etc can break down an explanation and present it in different ways tailored to the student's understanding. They can look at a student's work and even if it's incorrect, see what the student's train of thought was and understand what they were trying to do.

Our society already drastically undervalues that crucial, mind-accelerating work-- arguably the most important of all human endeavors, as everything else relies on it.

Glorified stochastic parrots spewing botshit are no damned substitute.