The reason I get so annoyed about people pitching LLMs as a way to 'democratise programming' or as end-user programming tools is that they solve the wrong problem.

The hard part of programming is not writing code. It's unambiguously expressing your problem and desired solution. Imagine if LLMs were perfect programmers. All you have to do is write a requirements document and they turn it into a working program. Amazing, right? Well, not if you've ever seen what most people write in a requirements document or seen the output when a team of good programmers works from a requirements document.

The most popular end-user programming language in the world (and, by extension, the most popular programming language), with over a billion users, is the Calc language that is embedded in Excel. It is not popular because it's a good language. Calc is a terrible programming language by pretty much any metric. It's popular because Excel (which is also a terrible spreadsheet, but that's a different rant) is basically a visual debugger and a reactive programming environment. Every temporary value in an Excel program is inspectable and it's trivial to write additional debug expressions that are automatically updated when the values that they're observing change.

Much as I detest it as a spreadsheet, Excel is probably the best debugger that I have ever used, including Lisp and Smalltalk.

The thing that makes end-user programming easy in Excel is not that it's easy to write code, it's that it's easy to see what the code is doing and understand why it's doing the wrong thing. If you replace this with an LLM that generates Python, and the Python program is wrong, how does a normal non-Python-programming human debug it? They try asking the LLM, but it doesn't actually understand the Python so it will often send them down odd rabbit holes. In contrast, every intermediate step in an Excel / Calc program is visible. Every single intermediate value is introspectable. Adding extra sanity checks (such as 'does money leaving the account equal the money paid to suppliers?') is trivial.

If you want to democratise programming, build better debuggers, don't build tools that rapidly generate code that's hard to debug.

@david_chisnall this isn't even a new phenomenon. Before there was vibe coding, there were "NoCode solutions". The problem is always the same: either the result is janky, limited, and/or not up to spec, or the person creating it has inadvertently become a programmer, with all the complexity that entails.

In the case of NoCode, this was mostly a way to underpay programmers, by not calling them that. I expect similar in the case of LLMs.

@sophieschmieg @david_chisnall And before that - COBOL

"COBOL is a language intended for use by non-programmers and, by extension, for non-programs"

@jackeric @sophieschmieg @david_chisnall Except COBOL works and was specifically designed for its problem domain. When FORTRAN was introduced, it got the same criticism - it deskilled programming. Like COBOL it was aimed at subject matter experts, not professional programmers.

COBOL is a weird and wordy language, an evolutionary dead end that nevertheless thrives in its ecological niche. Like FORTRAN, it is not a general purpose language but it is reliable and good enough at its job that it's difficult to replace. Neither are interesting to CS academics and that goes double for Excel.

I get it, it's a joke. No respectable CS grad would touch Excel, COBOL, or Fortran with a ten foot pole. They're inelegant and vulgar and contemplate because they haven't been displaced and eradicated by CS-oriented general purpose languages.

I am reminded that Jean Sammet (a principal developer of COBOL along with Grace Hopper) basically saved the ACM from insolvency in the early 1970s. Professional computing owes more to COBOL than it realizes.

@arclight @jackeric @sophieschmieg @david_chisnall Hmm? Fortran absolutely can be a general purpose language, it's just mainly used by certain sectors. And I'm a CS academic who did spend years on Fortran program analysis. Can't say I'm respectable though 😏
@md @jackeric @sophieschmieg @david_chisnall I mean, it mostly is now. It supports recursion, array operations, OO, pointers, and various forms of parallel programming without a ton of fiddly overhead. The standardized interface to C makes cross-language programming pretty straightforward and lets you link in a lot the native language doesn't support (or supports with difficulty). Still, I'd recommend a different language for general use, especially if you want a rich standard library, decent text processing, networking support, enums, or generics. It's still not a bad choice for numerically intensive code - it's low-magic, extremely stable, fast, and straightforward for subject matter experts to grasp. Saying good things about it in public will not get you invited to any parties however.
@arclight @jackeric @sophieschmieg @david_chisnall Yup, I'd rather that scientists continue to use Fortran than something dastardly like C++. I'm definitely not getting invited to any parties.

@arclight @jackeric @sophieschmieg @david_chisnall FORTRAN does have one point of CS interest - it (uniquely?) doesn't have the lexical analysis layer between the individual characters and the syntax, thus allowing for the famous

DO 10 I = 1.10

@sophieschmieg I think a lot of the low-code or no-code things were better because they didn't claim to be general solutions. They were good at creating business apps that worked like ten thousand other in-house business apps but with the small tweak specific to your requirements. Effectively, they factored out the common code and made it easy to create something that was the common bit plus a tiny extra bit.

In contrast, LLMs let you automatically generate a private local copy of all of that shared code, which is a maintenance nightmare and is completely unapproachable to non-programmer users because they can't tell the difference between the thousand lines of code that are copied and pasted and the dozen that are actually related to their requirements.

@david_chisnall @sophieschmieg If it only were so.

In my experience LLM produce often enough faulty enough code, that even in "dynamically typed languages" like Python manages to throw SyntaxError upon module import.

As I joked, the LLM have probably managed in the past weeks to produce more SyntaxError errors than me myself in the past decade or longer.

@david_chisnall
My personal worry is always, what happens all the cool uses of LLM that don't have output that need to follow a formal grammar, which can be used to validate the LLM work output.

Then I watch our corporate retreat where managers dreamy tell how quickly an AI can create lengthy position papers on topics they don't fully understand.

Surely after the paper is written by the LLM they'll go on to read the 1000s of pages of source @sophieschmieg

material to verify the position paper content.

So the moment the LLM starts producing huge amounts of work output, the question arises how are you supposed to make sure that the output is correct?

And that the LLM produce plenty of bullshit is indicated by the fact that current state of the art LLM optimized for coffee generation or not, manage to produce regularly syntax errors. That does not even include the code that is invalid at runtime.
@sophieschmieg @david_chisnall

@david_chisnall
So whenever somebody mentions AI/LLM, the first question is what is your result verification strategy. Because you need one for almost all use cases

@sophieschmieg

@yacc143 @david_chisnall @sophieschmieg That's the problem. Luckily (?) with software it gets tested at runtime. Unluckily, when it's a bridge it'll get tested when a load gets applied and it falls down.

Never forget, either: it's a plausibility engine. If you can verify the result, it'll lie to you, and when you catch it, it'll apologise and lie to you again until you're happy (the result is plausible) or you give up checking.

@Dss
Ah, but that's a slightly faulty reasoning.

No not all faults get tested just by usage.
There is a whole New world of bugs and failure modes when you put the "system" under stress. Be it your friendly port knocking software that spews some random bile at it, some nice people probing if you handle all parameters correctly, or simply race conditions that only show up under extreme load. Or the clusterf&%k decides not to scale as @david_chisnall @sophieschmieg

predicted and insured of a result in 24h it's still running after 3 days. Should have included more logging to show progress. Sigh.

So yes, there's are no cool videos of a bridge catching a harmonic frequency, that's right, but trust me, disasters software is quite capable to produce. (And the test coverage myth, it's tested by usage is painful. Trust me, I cursed rather loudly last week on that topic in my office.)
@david_chisnall @sophieschmieg @Dss

@yacc143 @sophieschmieg @Dss In particular, most security vulnerabilities appear in software that works for the common case. If it doesn't work at all, people don't use it and there's no impact from a bug. If it works well when no one is actively attacking it, it gets deployed in a lot of places. Software vulnerabilities are usually bugs that are triggered by some uncommon input (or, in some cases, common input but uncommon observations).

Testing can catch a lot of these if the tests are written by someone who understands the common failure modes of the kinds of software that's under test. Understanding is key here, and that's something LLMs lack.

@david_chisnall @yacc143 @sophieschmieg I agree that anything customer facing, or external, needs another level of care and testing. But no-one's* dumb enough to show an internal spreadsheet externally, surely?

*DOGE excluded, obvs!

@Dss @david_chisnall @sophieschmieg I've working in ETL and especially automated data acquisition (one could say web scraping, but the discipline is sadly a little bit wider) for the past decade or so.

Trust me, after 2-3 years you stop assuming anything, and the "why would any thinking being code that like this???" moves into the background.

Basically that "now I've seen everything" assumption has been kicked literally into the garbage bin too often.

Another thing that I've learned is that (we are in the academic adjacent space, so to say), Excel sheets are fucking bad as "databases" or "data exchange formats". Our CTO (when we were still a startup) literally designed a template sheet that interested partners could fill with their data if they wanted to give us their data (and didn't have it somehow published for us to scrape).

I've literally not seen in the dozens Excels that came back one f%c&ing sheet that didn't need custom code to be parsed.

What do people think an ISO 3166-1 alpha-2 country code like US/UK/FR/DE is? Deutschland is obviously the expected value for the cells, right? Or is it +49?

That was one fascinating experiment in "data entry", Excel sheets totally failing. You either need to code up a VBA app that requires a specific MS Office setup at the user or you get random data.

@yacc143 haha, amazing.

We had a VBA module called something like "Cleanuptext" that was used to parse every single field of input, because some of the engineers were very... Hard to help... But also because Access would do some crazy things where it broke itself on certain inputs! So in the end, *everything* got turned into A-Za-z0-9, plus a bit of punctuation.

@sophieschmieg @david_chisnall I question your premise that it's easy to see what and why Excel is doing something. From personal experience, it's often not even clear, why it stores certain fields as a specific type. Slight version difference, language settings, and the phase of the moon might influence it or not.

But yes, writing code is always the smallest part. You always read and try to understand code (even your own code) much more often. With debugging or not in mind. 1/2

That's why clever ways to show how well you know the language and how you are capable to shave off 35 bytes of code length are great (for an obfuscated coding competition, not real life).

Because the code will be read by sub-average colleagues who try to understand it. Something people tend to forget. 2/2

@yacc143
The person I'm most worried about reading my code is future me, though given I'm not a full-time, or even much of the time, coder I can assume that means it needs to be ready by a non-professional. I'm always happy when past me wrote a straightforward algorithm, or at least documented the hows and whys of the not straightforward algorithm. I'm always pleased when I can pick up code I wrote a decade ago and use or debug it without too much trouble.

@VATVSLPR Yeah, that's great.

Extra bonus if you can explain why you chose that path.

Extra extra bonus if you don't turn red in embarrassment too often.

That's something people have a problem to grasp, is in a way a huge difference between commercial and academic software development. In commercial development you often have to do with the developers HR hires for you. So nice safe languages, that make sure your developers keep all their fingers and their sanity.

In academic settings, especially at the higher levels, you tend to have a higher density of brainiacs, so of course a C++ lecturer is totally fine doing his PhD CS project in C++. 1/2

As my personal joke in the 1990s was, sure I have no problem leading a successful C++ development team. I just want to pick my team myself, and I'd start by raiding the CS department of their experienced C++ developers/lecturers (C++ complex language, especially in the 1990s not fully cooked yet, seasoned with the full power of C to dis-bowel the developers for one moment inattention). Now that's not how C++ was generally used at the time in the industry, hence the sea of C++ disasters. 2/2

@sophieschmieg The labor angle is a good one -- focus on the linguistic details over the underlying intellectual work has been a part of how some insecure people in this industry have been trying to keep out people who don't look like or think like them for years.

I think a lot of the "thought leadership" about who can be a programmer is based on observed differences (and thus downstream of all kinds of differences in privilege - education, non-disability, being included by default, etc.). I think alternate tooling (that works consistently!) with enough equitable treatment would make a lot of actual programmers who today don't think of themselves as such.

What non-programmers who opine on AI don't seem to appreciate is that programmers mostly end up preferring the ability to precisely specify behavior of a program

@david_chisnall

@david_chisnall What would a visual debugger look like for, say, Python?

Same as excel?

Some sort of infographic/flowchart generator?

pdb β€” The Python Debugger

Source code: Lib/pdb.py The module pdb defines an interactive source code debugger for Python programs. It supports setting (conditional) breakpoints and single stepping at the source line level, i...

Python documentation

@androcat Python is basically Smalltalk with ugly syntax and better FFI, so a starting point would be something like a Smalltalk debugger: everything is inspectable, you can create inspector windows for any object and directly manipulate it. Oh, and the code in Smalltalk (and Excel) is just a pile of objects (cells) and so you can dynamically modify any of it while the program runs.

Some of the more interesting work I've seen on debuggers for this kind of environment works by realising that reactive programming and watchpoints are basically the same concept: run some code when some variable changes. This lets you write arbitrary code in the language being debugged that runs when a particular variable changes (including things referenced by complex path expressions, so you can express things like 'starting from this root node of a tree, run this code any time a leaf node changes'. That lets you build live inspectors of program state and arbitrary traces for later inspection (and for turning into breaks on the next run).

@david_chisnall @androcat Did you ever see LightTable?
http://lighttable.com/
It was a visual (but not in the ordinary sense) programming environment. Take a look at the 'in action' video in the link above.
Every variable, every statement inspectable.
Light Table

Light Table is the next generation code editor. Light Table Connects you to your creation with instant feedback and showing data values flow through your code.

@sleepyfox @androcat Yes, it looked very much like an offshoot of the STEPS project from VPRI (of which I am a huge fan). I think the project is dead, which makes me sad.
@david_chisnall @androcat LightTable died years ago, the author Chris Granger went on to join (ironically) a no-code startup called Eve (which went bust four years later in 2018).
Even more ironically they went through multiple iterations of #NoCode or #LowCode products until they finally ended up with a text-based literate programming environment.
https://futureofcoding.org/essays/eve/
A Visual History of Eve

Are you looking for the real computer revolution? Join the club! Future of Coding is a podcast and community of toolmakers, researchers, and creators working together to reimagine computing.

Future of Coding
@androcat @david_chisnall I feel like that's a huge chunk of why Jupyter Notebooks have grown in popularity for non-programmers (thinking data folk, and science folk here) in Python. It's not quite as inspectable as a spreadsheet, but you can see it from there.
@david_chisnall all i'm reading is "copy excel"
@lritter @david_chisnall This gives me evil ideas for how to debug Tcl scripts.

@david_chisnall I once got into an argument with somebody pushing an LLM programming in Plain English thing and they insisted it wasn't horribly ambiguous.

I went to their docs and immediately pointed out their first simple example had like four entirely plausible interpretations for what the use might have actually meant. The person seemed to lack a theory of mind and ability to grasp that somebody might mean something other than what they personally intended in their example.

@wrosecrans @david_chisnall I bet! Deliberately misunderstanding code is impossible, or should be. English, not so much.

@david_chisnall I would argue that actual end-user programming got worse in the last decades. Excel remains popular (and programmers should know more about it), but the products right next to it seem to be out of favor.

I'm talking about Access, Filemaker or even the good ol' Clipper gang (from dBase to Visual Fox Pro). Focused, built-in storage, GUIs that might not be that great, but good enough for the specific use cases.

(Once again I'm blaming the client/server nightmare and the lack of any GUI standards for the inability of such tools to serve our timeline.)

What do you get from the vibe-coded version of this if you start out as a relative newbie? How do non-programmers pick their storage backend, for example? (Also I'm sure that somehow AI will get Kafka in there. It's bad enough in real life, so why would there be any relief from this. Someone needs to coin a word for this existential horror situation.)

And maybe it's just my particular bubble, but I don't see that much "real" end-user programming anyway. Some programmer trying new languages and frameworks (at lesat as new as the training cutoff allows), plus the semi-technical "founders". For the latter, it appears we're getting a horrible definition of "MVP" soon…

@mhd Access got a lot of flak because it did one thing very well and another very badly, and it was marketed as a tool for doing the latter.

I worked with a bunch of companies that had used Access to build UIs to things like stock-management systems. The GUI builder was drag and drop and each form was backed by an SQL query that you could create mostly visually.

The built-in DB's complete lack of most features of a DB was painful, but it was quite easy to migrate these things to using SQL Server.

With the rise of things like Ruby on Rails, it was easier to build more complex, more maintainable equivalents if you had an in-house Ruby programmer, the nice thing about Access was that a non-programmer could build a working GUI front end to a DB and a part-time DBA could make it something that could work long term.

Several of the companies I saw doing this had written their UI bits on Windows 3.11 and not really modified them up to Windows XP. They'd been solving real business problems for a decade with minor tweaks. Good luck doing that with a 'vibe coding' solution.

@david_chisnall @mhd I did that for about 2 years for a major multinational! We pushed Access, and VBA inside it, far beyond the limits! This was back when 30 users in the same Access database would corrupt the data - and we had 250+!

Have you ever seen a load balancing Access database front end, that sent users to different versions of the same database, then merged them later? Then cloned them back out every evening for a new database that management could use? We had literally hundreds of versions of the same database, under database control!

The maddest thing is, it worked! As Maslow said, though, "If you only have a hammer, everything becomes a nail".

@david_chisnall what good software engineers and system architects have to DREDGE from a requirements spec and PULL like TEETH from the know-nothing client. I do not miss that part of the job.

@david_chisnall Writing precise requirements is a very underrated aspect of engineering - irrespective of the domain.

There is a considerable effort needed to minimise the number of possible interpretations. Balancing it in a diverse global workplace with different cultural interpretations and different levels of language proficiencies is another challenge on top.

@david_chisnall And more importantly: teach people (how) to use those debuggers. For example, C# has amazing debugging tools including conditional breakpoints, easy ways to modify values while the program is paused and REPL-like live code execution... and I still see even senior developers who spend days littering their code with debug output because they're too afraid to even use those tools.
@dfyx @david_chisnall I don't want fancy breakpoints, I want invisible logging.
@jrootham @david_chisnall What do you mean by invisible logging?
@dfyx @david_chisnall Setting logging points that don't show up in the source. Preferably at a granularity below the line level.

@jrootham @david_chisnall C# can do that. You can configure a breakpoint to log something and immediately continue (which technically makes it a tracepoint). You can even combine that with a condition. See https://learn.microsoft.com/en-us/visualstudio/debugger/using-tracepoints?view=vs-2022 for an explanation.

And yes, in Visual Studio you can attach breakpoints/tracepoints at expression granularity by right clicking and selecting "Breakpoint -> Insert Breakpoint". VSCode and Rider probably have something similar.

Log info with tracepoints - Visual Studio (Windows)

Set tracepoints to log info to Output without modifying or stopping your code. Just specify an output string under the Action checkbox in Breakpoint Settings.

@dfyx @david_chisnall That's good. Unfortunately not my toolchain. I was mostly snarking back at the debugger fanboys in any event.

great points which I’m resisting jumping-off on novella-length tangents about πŸ˜†

Recent observation: using Copilot has actually improved how I explain technical problems. I’ve mostly worked solo or in small teams, so I felt my ability to communicate requirements unambiguously was underdeveloped; my technical communication is okay as I was a lecturer, but I often jumped into coding without externalising what needed to be done. Prompting forces me spell things out in plain English first πŸ€·β€β™‚οΈ

@Liamo Fully agree. I do data analysis not real "programming". Having a LLM write the python code for me is a god send, as I cannot ever seem to remember the plotting commands just right. Through the LLM I get those right.

As you mention, my prompting ends up being a reasonable fraction of the code, but at least I get a graph quickly.

@david_chisnall COBOL was meant to replace the developer and we all know how well that went...

@cosmic @david_chisnall I am not sure what you are referring to with that.

It has been vigourously touted that the verboseness of COBOL would make it easier to learn. This is of course mostly bullshit, but no worse than the bullshit accompanying promotions of other languages.

One advantage was however that it could thus lure non-programmers to get started, for purely psychological reasons, something needed because there was much too few programmers around in the 60s.

@mapcar @david_chisnall yeah that's more or less what I'm alluding to. A shiny new toy that is being advertised as making it easier for non-programmers to develop software with the outcome being more need for programmers to fix the mess made