Starting with the very specific: I do not think it was an accident that the xz backdoor's exploit chain started with a modified version of a third party .m4 file to be compiled into xz's configure script.
It's possible to write incomprehensible, underhanded code in any programming language. There's competitions for it, even. But when you have a programming language, or perhaps a mashup of two languages, that everyone *expects* not to be able to understand — no matter how careful the author is — well, then you have what we might call an attractive nuisance. And when blobs of code in that language are passed around in copy-and-paste fashion without much review or testing or version control, that makes it an even easier target.
So, in my capacity as one of the last few people still keeping autoconf limping along, I'm thinking pretty hard about what could be done to replace its implementation language, and concurrently what could be done to improve development practice for both autoconf and its extensions (the macro archive, gnulib, etc.)
On the subject of implementation language, I have one half-baked idea and one castle in the air.
The half-baked idea is: Suppose ./configure continues to be a shell script, but it ceases to be a *generated* shell script. No more M4. Similarly, the Makefile continues to be a Makefile but it ceases to be generated from Makefile.am. Instead, there is a large library of shell functions and a somewhat smaller library of Make rules that you include and then use.
For ./configure I'm fairly confident it would be possible to do this and remain compatible with POSIX.1-2001 "shell and utilities". (Little known fact: for a long time now, autoconf scripts *do* use shell functions! Internally, wrapped in multiple layers of M4 goo, but still — we haven't insisted on backcompat all the way to System V sh in a long, long time.) For Makefiles I believe it would be necessary to insist on GNU Make.
This would definitely be an improvement on the status quo, but would it be *enough* of one? And would it be less work than migration to something else? (It would be a compatibility break and it would *not* be possible to automate the conversion. Lots of work for everyone no matter what.)
Suppose that's not good enough. Bourne shell is still a shitty programming language, and in particular it is really dang hard to read, especially if you're worried about malicious insiders. Which we are.
Now we have another problem. The #1 selling point for autotools vs all other build orchestrators is "no build dependencies if you're working from tarballs," and the only reason that works is you can count on /bin/sh to exist on anything that purports to be Unix. If we want to stop using /bin/sh, we're going to have to make people install something else first, and that something else needs to be a small and stable Twinkie. Python need not apply (sorry, Meson).
What's small and stable enough? Lua is already too large, and at the same time, too limited.
There's one language that's famous for being tiny, flexible, and pleasantly readable once you wrap your head around it: Forth.
If I had investments to live off, I would be sorely tempted to take the next year or so and write my own Forth that was also a shell language and a build orchestrator, and then have a look at rewriting Autoconf in *that.* This is the castle in the air.
Side bar 2: Let's table the whole "shouldn't everyone build from git nowadays?" discussion. I'm quite sure the xz insider could've found a way to hide the stage 0 exploit in a checked-in file. If you care about ways to make the output of "make dist" verifiable and reproducible, and to facilitate building from VCS checkout for those who want that, we're actually having a productive discussion about that on one of the autotools mailing lists right now.
(Not sure which list — I sort them all into one mailbox — and I have to warn you that several other less helpful conversations are happening under the same subject line.)
Moving to the more general.
I said this over on the autoconf lists earlier today: just as I think it is a mistake to focus on the stage 0 exploit having been concealed by not checking it into the VCS, I also think it is a mistake to focus on the next few stages having been concealed in a binary file. There are binary files that are naturally editable and auditable as themselves (raster images, for instance) and there are text files that nobody wants to look at at all (ever tried to fix a merge conflict in an SVG image?)
A more interesting line to draw, IMO, is between code and tests. I feel quite confident in saying that the files written to $prefix by "make install" should never need to have any sort of dependence on the project's test suite, and that is something that ought to be possible to detect mechanically (the biggest challenge is determining what files of the source repo are exclusively part of the test suite).
Last bit. Community, sustainability, and trust.
The early free software movement (1983–1994 give or take) was, as I've heard the tales, consciously revolutionary, and, as revolutions often do, it ran on the spare time of relatively young people with time and energy to spare.
I came on the scene in 1997, right about the time it became reasonably possible to run Linux as your only desktop OS if you knew what you were doing — or, to put it another way, right about the time the original goal of the GNU Project had been achieved.
Like many other revolutions, GNU had no answer, and still doesn't, to the question: now what?
This is not the only reason the young, energetic revolutionaries of 1997 are now the exhausted maintainers of an archipelago of individual "projects" that sort of add up to a computing environment that one might fairly describe as "the worst (except for all the others)". But I think it's an important reason.
Side bar 3: In the middle 1990s someone — either Eric Raymond or Guy Steele — wrote as part of the "Portrait of J. Random Hacker" appendix to the Jargon File
> [Among hackers] racial and ethnic prejudice is notably uncommon and tends to be met with freezing contempt.
This was not true even at the time, and twenty years later ESR was cheerfully making common cause with Vox Day and the Sad Puppies.
I'm a white guy (albeit some of my grandparents weren't). I already knew how to program when I got to college. If I'd made different choices in the early 2000s, I could very well now be sitting on enough investment income to take a sabbatical and invent a new shell language.
When we look around and say "where do we find the helping hands we so desperately need?" we must recognize that part of the problem is that hacking was never as inclusive a club as we claimed.
(This sidebar is not *only* a response to the commenters who saw a name like "Jia Tan" and immediately started hating on China as a whole.)
@zwol
I'm not smart enough to hang here so just gonna add
> toasting in an epic bread
and if you start a patreon to fund your forth project I'm in for ten bucks