Before AI, I felt like I could review PRs and, based on whose code I was reviewing, I could be more trusting and casual with my review, since I knew they usually wrote good code and paid attention to details. Now, with AI, no one pays attention to details, so I have to deeply scrutinize every PR, which takes twice or 3x the effort.
AI agents almost always add things that arenāt necessary, or they follow anti-patterns based on the surrounding legacy code context.
@ramsey Just let an AI do the review.
In all honesty:
* If you don't want to accept AI PRs, say so and reject all AI looking PRs without further looking at it.
* If you (have to/want to) accept AI PRs: Go full in and let the AI do the "review". REquest full documentation so that you can run End-To-End tests based on the requirements. Fuck all human based measurements that we used to hold up.
I don't think there is a sustainable way in between... š
@heiglandreas @ramsey This is becoming the norm at my workplace for folks that can bypass the "[Human] Review Required" check.
It is incredibly frustrating to watch so much slop go out the door. I'm no longer rubber stamping the PRs so instead they get someone else under them to Approve and out it goes. It's making me seriously consider withdrawing entirely in the next few months.
@heiglandreas @ramsey "Management wants" has become the dissolution of responsibility for everyone up and down the chain. š„“
The rift between those who care and those who need a paycheck, that has long existed and has been getting worse the more money has been pumped into the industry, has been ripped wide open, and I suspect what we're going to see, or need to see, is the former gathering together at their own companies and projects while the latter ... š¤·āāļø
@ramsey I have the same experience. Reviewing AI-generated/assisted PRs takes about five times as long as reviewing a human created PR as there is just so much wrong with them.
In nearly all cases, I could have written a better and cleaner fix myself in less than half the time the initial review cost me.
And then the PR still has to go through additional review rounds before it's merge-ready.
It's a time-sink on open source's most precious commodity: maintainer time.
And this is unacceptable.
@jrf_nl @ramsey the responses of the "authors" are even worse. I closed multiple pr's last months something I didn't do for a very long time.
But the arguments against closing are so offending you that I have been thinking about stopping with reviews.
I nearly lost the joy in working on my projects. Because of the shit posts I got.
@jaapio @ramsey I feel your pain.
I often feel my review is fed straight back into an AI and the Op posts the reply. Makes me want to just go sit in a corner and cry.
If you think it would help, feel free to re-use the snippet I added to the CONTRIBUTING file for PHPCS: https://github.com/PHPCSStandards/PHP_CodeSniffer/blob/4.x/.github/CONTRIBUTING.md#do-not-submit-ai-generated-prs
That snippet has been there about two years now and not sure if it is related, but compared to other projects, I feel PHPCS receives relatively few AI generated PRs.
@derickr But I think this one will give you an impression: https://github.com/phpDocumentor/guides/pull/1295
Especially the talking with a bot, not a human... Long walls of text that say nothing. And maybe something I take too personal, the suggestion that "I don't care". Because all the time they spend...
Talking about walls: https://github.com/TYPO3-Documentation/render-guides/pull/1143
In the moment, responses might feel like an attack, but reading them back. Most of this is just a waist of time.
"You don't care Because all the time I spend..." should qualify for an insta-block (unless they also provided a 50,000ā¬ submission without strings attached).
Sadly those fuckers will set up the next account with another throwaway emailaddress in minutes and send in the exact same PR 35 times.... š¤¬
š«
@ramsey Yeah, Iāve noticed a distinct drop in quality since coworkers started using LLMs.
Itās hard to be sure, because itās made me judge every PR more harshly anyway. But I notice some distinct patterns, like specifying default values unnecessarily.
And usually a failure to notice appropriate opportunities for abstraction (because hey, the LLM is so good at boilerplate that we donāt need to worry about that any more, right?)
We had a couple of big PRs of ānice-to-haveā features that were too low-priority ever to get done. Vibecoding meant they were initially written in a couple of hours ā but then they took literal weeks of back and forth in code review to get them into a tolerable state.
@benjamineskola @ramsey true. What I notice is that LLMs introduces a lot of duplicated code. Making the mental model harder and harder to understand, in case an actual human needs to read the code.
So a good example might be to define one static constant value and reuse that value across the file. Ai will duplicate that constant for each method or worse, create magic values (aka without any variable definition).
So Ai loves generating the same code.
@melroy @ramsey Yeah. People keep saying āitās great at handling boilerplateā. The flip side of that is that it creates boilerplate where none was necessary.
I think part of the value in doing code review for my teammates is that I start to see repeated patterns where we could abstract things out. An LLM never notices that; and automating the repetition shelters the developer from noticing it, too.
... I feel your pain. Same with my work, but no big public repos.@ramsey ok then we have the same problem 
.
I told my team leader that I don't mind him calling me at 23:55 to fix production issues š„. However, since we started accepting AI code, I have told him that he can call Claude when there is a problem with production.
The same applies to reviews. We are currently discussing the introduction of transparent AI disclosure. Has AI been used? What was it used for? In addition, we have considered removing reviews completely. This would make developers 100% responsible for the code they merge, and it would also prevent me from approving AI code. If you merge that, you alone are responsible for it.
As a final rule, the reviewer is always right. There is no discussion.
Ben š±
Ben Ramsey
Alerta! Alerta!
Russell
Sarah Savage
Juliette
Jaapio
Derick Rethans
Ein āPrimatā
ben
Anders Anni
Melroy van den Berg
0xC0DEC0DE07EA
Lukas Rotermund