Nolan Lawson

I think what a lot of AI critics are missing is that they're judging an LLM by its first draft. This is *not* what terrifies me about these machines.

What terrifies me is that you can ask them "find bugs in this PR." Or "find performance flaws." Or really anything.

Then have 3 agents (with different models ideally) vote on the result. Then have another fix it. Repeat until all bugs are clean.

If you haven't tried this experiment then you haven't reached the dark night of the soul that I have.

Mar 7 at 2:08amWeb
Show threadNolan LawsonMar 7

I've definitely had PRs that required like 5-10 rounds of reviews like this until the PR was sparkling clean. But the scary part is that it didn't really need me in the loop at all. Honestly you just needed a monkey to keep typing "make it better" and eventually it would.

If that entire process burns a ton of energy and water then that's all atrocious, but it's likely still less than an engineer's salary. And my body requires energy and water too. This is what scares me ultimately.

Show threadAndy LuhrsMar 7
@nolan This GIF has been the most apt feeling in many of my AI-centric chats.
Show threadNolan LawsonMar 7
@aluhrs 💯
Show threadStéphan KochenMar 7

@nolan I think the 'humans require energy too' is kind of a dark train of thought. And we're making ourselves dependant on corporations that will happily explore that fully, because Sam Altman made that same exact comment recently.

In this specific context, I see the positives, but I want control and transparency of environmental costs.

Show threadNolan LawsonMar 7
@kosinus You're right, the better path is to focus on one's humanity. I've been reading more fiction and going to more live performances recently. I'm finding less and less art and humanity through coding.
Show threadMx. Aria StewartMar 7

@nolan Yeah, though that line of thinking is a false equivalence, since the point of the water and energy for us is to live, and ideally live well. The point of work ... is not. It’s a step to enabling each other to live and work well.

We can't equate human resource use like that because the purpose is different.

Show threadNolan LawsonMar 7
@aredridel You're right, it's a dehumanizing comparison. I think I may have been dipping too deep into despair recently.
Show threadMx. Aria StewartMar 7

@nolan Yeah :| It's so easy.

That idea is also ambient, because that's a thing that Sam Altman said too. It's easy to end up parroting his words. It's the key losing the plot we're in danger of. The goal is good lives for people — which to be clear, includes achievement — but if we start measuring the achievement instead of the good lives, we go to the bad place very quickly.

Show threadNolan LawsonMar 7
@aredridel You're right; there is a lot of language from that camp dehumanizing the humans and humanizing the robots… both of which are wrong.
Show threadMx. Aria StewartMar 7
@nolan Yup. It's a weird spot, and hits humans right in the middle of a heap of biases, so it's doing some super strange things.
Show threadEugen RochkoMar 7
@nolan What you're describing is not a skill or craft, it's a gacha machine. It's gambling. You're hitting spin until you win something. And it relies on similar code and programs being in its training data. It's copying from Stack Overflow with extra steps. It won't solve novel problems.
Show threadbeatrix bitrotMar 7
@Gargron @nolan yeah, i think that's what's bumming him out
Show threadEugen RochkoMar 7
@bea @nolan Bea?!
Show threadbeatrix bitrotMar 7
@Gargron @nolan lmao yeah, hi, i'm still alive isn't that wild?
Show threadNolan LawsonMar 7
@bea @Gargron Exactly, yeah. A lot of software is not terribly novel. That's exactly the weakness that these tools are exploiting.
Show threadDanny PageMar 7
@Gargron @nolan @bea Having a good validation mechanism lets you stop gambling when you finally land on a good roll of the dice. And software engineering has a lot of those thanks to unit/integration tests, linting/complilers, and just plain checking to see if the code outputs something that you want.
Show threadEugen RochkoMar 7
@danny @nolan @bea It would be funny if it wasn't tragic that the idea of modern software development is to hit the randomizer button until something vaguely correct comes out instead of just knowing how to do something and doing it intentionally.
Show threadNikita KaramovMar 7
@Gargron @danny @nolan @bea yeah. I remember my university professors talking about how stupid and laughably inefficient bogosort is, and now everyone's in my feed saying how much they love bogosort and how it saves them time 🤡
Show threadEugen RochkoMar 7
@nolan @bea The problem is of course the deskilling. Once there *is* a novel problem that's important to solve, the world will be in a heap of trouble. There is this miconception that knowledge is on a constant upward trajectory, but actually it gets lost all the time. Nobody knows how to make cassette players as advanced as we used to because this technology was not considered important enough to be kept around.
Show threadNolan LawsonMar 7

@Gargron @bea Yeah, this reminds me of this excellent talk: https://www.youtube.com/watch?v=pW-SOdj4Kkk

I see it less as a tragedy to be prevented, though, and more just an inevitability of how human society accumulates complexity. Joseph Tainter's "Collapse of Complex Societies" also touches on this.

Jonathan Blow - Preventing the Collapse of Civilization (English only)

Jonathan's talk from DevGAMM 2019.https://www.youtube.com/c/DevGAMMchannel

YouTube
Show threadBálint SzilaksziMar 7
@bea ohai ltns!
Show threadbeatrix bitrotMar 7
@szbalint hello! yes it's been a minute!
Show threadEugen RochkoMar 7
@bea @nolan Yes, I feel like I haven't seen you since 2019 or so, nice to see you're still around.
Show threadbeatrix bitrotMar 7
@Gargron @nolan i was gone for a long time, but i came back. it's been nice, even though i don't have a tonne to say these days. nice to see you're still around and apparently haven't been driven completely mad
Show threadG1itchbitMar 7

@nolan @Gargron While I agree you, neither users nor businesses care about the skill or the craft. Never have. They get mad when the magic breaks and happy when the magic is fixed.

You may be able to find those that care, maybe some rich eccentrics. Communing with other devs perhaps.

Alternately, all these frontier models are operating at a loss. Maybe it won't be cost efficient after all once it stops being VC subsidized.

Show threadTorbMar 7
@glitch @nolan @Gargron Considering how users have been complaining for years about the broken or user hostile state of the digital state of what they use I absolutely think they care. Not for it’s own sake, but because they want things to reliably work.
Show threadMatt CampbellMar 7
@torb @glitch @nolan @Gargron Case in point: I was at my favorite karaoke bar yesterday evening. I hang out with a group of mostly older folks (60s and up; I'm in my 40s). One regular, Kathy (70s), talked about a bug she found in Facebook on her Android phone and tablet: When someone posts text on a colored background, you tap to open the post, and the text *isn't there*. She tried googling the problem, clearing a cache, rebooting, etc. It clearly unsettled her. And that's a fairly benign bug.
Show threadMartin HamiltonMar 7
@Gargron @nolan StackOverflow As A Service: fails gradually, then suddenly
Show threadSimeon NedkovMar 7

@nolan The biggest drawbacks of this tech, for me, are the non-tech aspects/consequences and second/third order effects.

I expect it to remain problematic, even if it one day can one-shot everything I throw at it.