I think this will be a watershed moment in tech similar to Elon's layoffs at Twitter in 2022. AI coding agents crossed the threshold in December and this is the beginning of the fallout.
"we're not making this decision because we're in trouble. our business is strong. gross profit continues to grow, we continue to serve more and more customers" - Jack Dorsey
and yet folks using these tools still can't successfully make contributions to open source projects and more and more open source projects are crafting rules to deal w/ LLM slop contributions: https://github.com/melissawm/open-source-ai-contribution-policies
These can both be true, some projects lend themselves to these tools working well but we are clearly seeing many many projects do not.
We saw from the CCC compiler story that while it could make a compiler it also got to the point where it was no longer maintainable and this was long before it was even close to being a viable product: https://harshanu.space/en/tech/ccc-vs-gcc/
There is a line, where the tools do very well and where they can't hack it. No one has quantified where that line is but the line does exist. Compilers and large open source projects don't seem to be on the right side of the line. Prototyping, hacking, web services, refactoring APIs and scripting are definitely on the right side of the line.
We are going to keep seeing waves of dread and euphoria like this and we need to stay grounded in objective evidence and not succumb to hype.
A list of policies by different open source projects about how to engage with AI-generated contributions. - melissawm/open-source-ai-contribution-policies
If the tools were that good than just feeding in the requests from the reviewers should be sufficient. If not than they are useless for large mature projects which usually are composed of small targeted fixes.
I don't agree they have to keep getting better. The LLM model is by its nature limited. You can't tweak yourself out of the limitation of statistical inference. It is like a asymptotic function, if you are near the asymptote you can keep giving it larger and larger numbers but it can still only approach the asymptote. Where the limit is an open question but like all AI methods for the past five decades there is one.
@shafik @carnage4life I agree that the LLM ability curve is probably asymptotic, I'm just not sure why you would think we're approaching the asymptote.
I also agree that AI isn't good enough yet to fix issues autonomously and reliably. But with someone who can read code and who is familiar with the codebase supervising, it can get a lot more done in a lot less time than coding by hand. The problem with these AI slop PRs is that people are expecting the project maintainers to be that AI supervisor, which is unfair and just creates more work for them, not less.
Dare Obasanjo
Shafik Yaghmour
Gunther