AlexJun 12, 2023
Nolan Lawson

I think this essay by @baldur and his book "The Intelligence Illusion" may have converted me to a full-blown AI skeptic. In particular, for things like coding completion / code generation.

- https://softwarecrisis.dev/letters/ai-and-software-quality/
- https://illusion.baldurbjarnason.com/

Modern software quality, or why I think using language models fo…

How to make better software with systems-thinking

Out of the Software Crisis
Jun 11, 2023 at 7:10pmWeb
Show threadNolan LawsonJun 11, 2023
The argument that resonated the most with me is that high-quality software needs living, breathing human developers with a mental model of how the software works. If you're taking huge code blocks spit out by an AI and injecting it into your code unread (or largely unread, as the tool encourages you to do), then you're effectively taking on a dependency – one that's not understood by anyone on your team, of dubious quality, and frozen in time with no security updates or bugfixes ever.
Show threadNolan LawsonJun 11, 2023
After several years of this, you basically end up with a legacy codebase that is poorly understood by anyone on your team. Which – to be fair – you might end up with anyway due to attrition, reorgs, etc. But why would you want to wish that scenario on yourself?
Show threadNolan LawsonJun 11, 2023

To be fair, though, I have a lot of biases here:

- I haven't used Copilot or anything similar
- I used to work in ML when it was mostly dumb Markov chains, I experienced the limitations of the tech before LLMs hit the scene
- I'm a fast typist, it's one of my biggest advantages as a developer TBH

Also I'm sure the tech has its applications (e.g. Baldur makes a good case for transforming/converting text). I just think few are wondering about the long-term downsides of relying on these things.

Show threadHipsterBrownJun 11, 2023

@nolan text transformation and summation is exactly what I’ve been trying out with the various models. Ingesting a bunch of docs and querying that knowledge base with natural language and context.

It’s also been a “fine” approach to getting a starting draft for various writing needs. There’s plenty of room for improvement.

Show threadwestbrookJun 11, 2023
@nolan The difference between this and normal legacy projects is likely the it actually wouldn’t take “several years”, if you’re doing this on a regular basis. In that case, you could be gifting yourself a foreign legacy codebase once a month or so. 😰
Show threadAlec PerkinsJun 11, 2023
@nolan I’ve found Copilot useful as an ethereal framework of sorts, where it can conjure up a pretty good solution to a common problem (that maybe should live in a library). It’s also helpful for any repetitive boilerplate in things like test setup. But for any business-specific logic it’s pretty useless and sometimes a hinderance.
Show threadAlec PerkinsJun 11, 2023
@nolan it kind of feels like pairing with a dev who has seen a lot, but doesn’t know or care much about your objectives. Or like pairing with every Stack Overflow responder.
Show threadMarcus JenkinsJun 12, 2023
@nolan it's software architecture and disciplined development with TDD and Continuous Delivery. AI will master this. For sure. Soon.
Show threadnfrgrtJun 11, 2023

@nolan @baldur

It is astonishing mimicking my style, quickly understands what I'm up to (even "odd" things) and sometimes comes up with interesting ideas.

That said, if you let it generate tests & business code, you might get bullshitted twice while getting things "green".

I highly recommend reading and understanding its output. And reading, understanding and debugging other people's code is already the biggest drag in our work. So mixed feelings, if it's worth a dime.

Show threadsnowyfoxJun 11, 2023

.

from citations:
『No Silver Bullet: Essence and Accident in Software Engineering』 by Frederick P. Brooks, Jr.
http://worrydream.com/refs/Brooks-NoSilverBullet.pdf

Show threadAndrás N.Jun 13, 2023
@nolan @baldur Great article! Thanks for the recommendation.