People keep assuring me that LLMs writing code is a revolution, that as long as we maintain sound engineering practices and tight code review they're actually extruding code fit for purpose in a fraction of the time it would take a human.

And every damned time, every damned time any of that code surfaces, like Anthropic's flagship offering just did, somehow it's exactly the pile of steaming technical debt and fifteen year old Stack Overflow snippets we were assured your careful oversight made sure it isn't.

Can someone please explain this to me? Is everyone but you simply prompting it wrong?

It's a good thing programmers aren't susceptible to hubris in any way, or this would have been so much worse.

You know, it isn't even that tools like this are useless. There are absolutely things they could be good at. I've personally seen Claude find stupid little bugs you'd spend an hour figuring out and hating yourself for afterwards with great efficiency. I tried the first iteration of Copilot, back when it was just an aggressive autocomplete, and while I had to stop using it because it was overconfidently trying to finish my programs for me without being asked, it was great for filling in boilerplate and maybe even a couple lines of real code for the basic stuff. We have models nowadays that are actually trained to find bugs and security issues in code rather than having the entire internets thrown at them to produce something Altman & Amodei can sell to the gullible as AGI.

But there's the problem. The technology has been around for a while, we have a good idea of what it's good for and, more importantly, what it's not. "Our revolutionary expert system for finding bugs in your code" isn't nearly as marketable to the general public, and the CEO class especially, as "our revolutionary PhD level sentient AI that will solve all the world's problems if you only give us another couple trillion dollars, and also wants to be your girlfriend." And so we get Claude and ChatGPT and RAM shortages and AI psychosis and accelerated climate change instead of smaller, focused models that are actually good at their specialist subjects. Because those don't produce as much shareholder value.

@bodil I liked @mmasnick's take on how mayyyybe there's a silver lining in code-generating that it can help re-democratize personal computing in which it's not the personal computer but also the software can be customized and home-grown.

I like to think that sammy boi is out there, trying to buy up the world's complete silicon wafer production because he spends his sleepless nights dreading gen AI breaking loose of his ilk's corporate capture.

I'm sure many of us won't gleefully march into local-AI boosterism without addressing the (open-weight) elephant in the room, maybe one way truly open & fair models will leave the fairy realm of the Mozilla Foundation "Wouldn't It Be Cool..?!!" list.

Like, waiting for the "AI bubble to pop" is like hoping for an alien invasion: all it will bring is pain and destruction with no clear "ok, what now?" that follows. I like the _hopefulness_ of his perceived trajectory and I truly hope we get there before we split the planet in half. 😶

@flaki
Software has always been homegrowable and customizable. Society chose to reject people actually customizing it by mass marketing computers that have increasingly complex requirements for being "useful". (Hell, even the good old C64 is packed with proprietary bits.)
LLMs democratize nothing, local or not. Good docs, relative simplicity and community do.
@bodil @mmasnick