AI is making us write more code. That's the problem.
I analyzed research papers on AI-generated code quality. The findings:
→ 1.7x more issues than human-written code
→ 30-41% increase in technical debt
→ 39% increase in cognitive complexity
→ Initial speed gains disappear within a few months
We're building the wrong thing faster and calling it productivity.
@mlevison My anecdotal guess is that people are operating above their own abilities.
If I understand a code base and its architecture, dependencies, business logic needs, etc. Then I can ask an AI to make a targeted change and verify it. It’s basically avoiding some typing.
But if I don’t understand those things I don’t actually know how to ask or how to verify it.
Even worse if I wouldn’t be able to write it in the first place, how can I ask for it and verify it?
Basically, how can I successful operate above my own knowledge and ability? It’s like asking for a piece of text written in a language I don’t speak. How can I have a clue what it says?
I think there are so uses for assisted development. I’m less sure an LLM is the way to achieve that.
Worst part, people aren’t learning anything from all this. There’s no skill and knowledge being built:(
Imho, ymmv, etc.
Also those are some numbers, yikes!
@yon I have lots of theories as well.
Some of my friends have experience that contradicts the data. Ideally we would measure the quality of the code from people doing this well and compare to the baseline.
Until we do that we have to assume we’re no better than the baseline.
Mark Levison
四