Craig Stuntz3d ago
Edwin Brady

Today, thanks to a former PhD student, I learned that LLMs are terrible at generating Whitespace code: https://arxiv.org/pdf/2603.09678

Also today, I was invited (and I am accepting) to join a group on redesigning our undergraduate curriculum, which includes discussions on our choice of first programming language.

I am not saying that these things are related. I just happen to be mentioning them both at once because I saw them both today. You get a lot of characters in one toot after all.

What?

Mar 20 at 9:25pmWeb
Show threadgaytabase3d ago
@edwinb they're not great at other code either
Show threadEdwin Brady3d ago
@dysfun Who would have etc
Show threadgaytabase3d ago
@edwinb whitespace of course is the easiest language to verify
Show threadSlightlyCyberpunk3d ago
@dysfun @edwinb I'm going to once again take this opportunity to rant about that time this motherfucker I work with spent *an entire day* trying to get the LLM to create a shell script to select one column from a bunch of delimited text files (maybe 50k lines in total), and I came in after he'd already started executing that program and wrote and executed my own script to do the same task before that slop was even 10% done!
Show threadFinn2d ago
@admin @dysfun @edwinb are you aware there's also a shell utility for this (cut)?
Show threadSlightlyCyberpunk2d ago

@finn @dysfun @edwinb Yeah, but cut alone isn't gonna do it when the files use multi-character delimiters and each field is a key/value pair...which is probably why the AI needed ten damn hours to get it done, because it's marginally more complex than what you can copy/paste off Stack Overflow lol

Two cuts and a pipe probably could have done it; I think I went with sed though lol

Show threadFinn2d ago
@admin @dysfun @edwinb oh yeah that'll do it!
Show threadWellsiteGeo3d ago
@edwinb You will require students to submit their Whitespace submissions double-spaced, or single spaced?
Show threadShadSterling3d ago
@edwinb hmmm. Now I want to see an intro programming course using an in-house language, and a later course which makes breaking changes to the language every semester
Show threadColin Watson2d ago
@ShadSterling @edwinb When I was at Cambridge they used ML on the explicit basis that people were unlikely to have learned it as teenagers and so it created a level playing field. Feels like it rhymes with the current LLM problem, at least.
Show threadJan de Muijnck-Hughes3d ago

@edwinb good luck with the curriculum discussions! The dicussons are actually interesting as it forces us to think outside our comfort areas. CS is a cyber-physical socio-technical theory and practice based subject. At St Andrews it’s compounded with the major minor tertiary module structure. You need the polder model if discussions are going to work.

Related side note, I remember years ago hearing a discussion in St Andrews about first languages. Fascinating the arguments were.