LucomoMay 23

AI is a horrible tool in software development.

A team leader explains the problems in this post on Reddit:

"AI Slop PR's are burning me and my team out hard"

https://www.reddit.com/r/ExperiencedDevs/s/MeNimUXpsy

Show threadSimon
@Lucomo gotta hit those 5k line PRs with one of these.
May 26 at 6:56pmWeb
×
LucomoMay 23

AI is a horrible tool in software development.

A team leader explains the problems in this post on Reddit:

"AI Slop PR's are burning me and my team out hard"

https://www.reddit.com/r/ExperiencedDevs/s/MeNimUXpsy

Show threadRyan J. YoderMay 23

@Lucomo
I personally haven't experienced this a lot but it sounds like something you just have to correct Junior devs not to do. And it will take time. Don't let a junior dev get away with pushing code they don't understand.

If senior devs did this, I'd be very irritated and I'd have a serious conversation with them about it.

Show threadJared White (🏳️‍⚧️ ally)May 23

@ryanjyoder @Lucomo this very much sounds like a "you're holding it wrong" argument.

The fact that these tools do allow junior devs to submit wild & ungrounded PRs is itself a major, major problem.

Show threadRyan J. YoderMay 23
@jaredwhite @Lucomo
Maybe? But they already do that when they copy code from stack overflow without understanding it, which has to be corrected and they figure it out eventually.
Show threadAsh_CrowMay 24
@ryanjyoder @jaredwhite @Lucomo PRs copied from StackOverflow don't change 5k+ lines of code
Show threadÐДѷє ۷ǿȵ ຣMay 24
@Ash_Crow @ryanjyoder @jaredwhite @Lucomo if a junior dev would do this, they would have a very serious talk. This is not just copying code you don’t understand, it’s deliberately wracking the system
Show threadTobias DenkingerMay 24
@ryanjyoder
Cobbled together code from StackOverflow is relatively easy to recognize. AI slop is not (while being even easier to generate). LLMs are specifically trained to generate hard to recognize code (or text in general). This makes the "bullshit PRs problem" at least one order of magnitude worse.
@jaredwhite @Lucomo
Show threadRyan J. YoderMay 24
@denki @jaredwhite @Lucomo
I may be coming at this from a different perspective. I'm not working with any engineers that are trying to bs me. So it seems easy enough to detect and correct this behavior. But if these PRs were being opened against an open source project by a new contributor, I can see how it'd be very hard to review.
Show threadTobias DenkingerMay 24

@ryanjyoder
I am also not working with any engineers that try to bs me either. But we are not (yet) using LLMs for code generation in our team.

Judging from my LinkedIn feed, however, management may try to encourage the use of LLMs to "increase productivity". So the engineers might even be encouraged to submit bs.
@jaredwhite @Lucomo

Show threadRobin AdamsMay 24
@jaredwhite @ryanjyoder @Lucomo I don't think this is a junior dev using AI to generate PRs, I think this is one of the places where they've allowed the AI to submit bug reports and PRs directly.
Show threadRyan J. YoderMay 24
@robinadams @jaredwhite @Lucomo
Ah. Oof. That's quite different.
Show threadAlexander The 1stMay 24

@ryanjyoder @Lucomo Speaking aside a former junior dev, we won't create additional files that seem to serve no purpose, and create 5K line code changes.

They tend to be smaller, on account of being smaller features given to us to work on.

Show threadBradley SchaeferMay 24

@ryanjyoder @Lucomo I have already seen this. It’s not always obvious to untangle since it can appear plausible - it can take more expertise and effort to debug than it would have to write up-front.

I predict a lot of software is going to become janky and unmaintainable as early adopters have false confidence.

Show threadJonathan YuMay 24
@Lucomo Regardless of tools (IDEs, StackOverflow, coding tools), I think authors of PRs are responsible for PR contents. And the burden of proof that a change is safe and desirable is on the author, not the reviewer. That's my current opinion, anyways :)
Show threadmattgMay 24
@jawnsy @Lucomo for sure. "I'm not reading 5k lines. Try again in smaller chunks," is a perfectly acceptable PR review comment.
Show threadSpazcosoftMay 24
@Lucomo the whole world sucks now. :(
Show threadwb x64May 24
@Lucomo "I used AI to do it" really needs to sound more like "I asked my younger cousin to do it"
Show threadShredd Tone  May 25
@wilbr @Lucomo Exactly 👆
Show threadMark OtwayMay 24

@Lucomo

People in the comments suggesting, completely seriously, that the solution is to use an AI reviewer to review the PR before a human looks at it.

JFC.

Show threadfeldMay 24
@Lucomo lets be honest, if they can't be bothered to review the code before committing these same people would have been copy and pasting directly from stack overflow

Fire them all
Show threadAbieMay 24
@Lucomo I first read PR as Public Relations... Is it Product Review?
Show thread13 barn owls in a trenchcoatMay 24
@temptoetiam @Lucomo Pull Request - without getting into the details of how Git works, it's when a contributor submits a proposed change in a software's code for the team maintaining it to look at.
Show threadAbieMay 24
@HauntedOwlbear @Lucomo thank you!
Show threadMachine Lord ZeroMay 24
@HauntedOwlbear @temptoetiam @Lucomo Thsnks, was looking for this
Show threadAlexander The 1stMay 24
@HauntedOwlbear @temptoetiam @Lucomo As I up, this is Git-specific terminology, so you might also just hear "Code review" instead, or alongside it. Same basic idea, some details are different.
Show thread🇵🇸 Álvaro GonzálezMay 24
@temptoetiam Pull request. That's GitHub lingo. GitLab, for instance, has merge requests.
Show threadTobias DenkingerMay 24
@temptoetiam
Pull Request.
@Lucomo
Show threadgazspMay 24

@Lucomo It doesn't have to be this way.

Yes, use AI to help you solve issues in your code, but don't let it write missions critical stuff. I might get it to write unit tests or method comments, but I'm not going to let it to touch any key auth code, for example.

We use Claude / Cursor at work and we don't have these issue (we also don't have any junior devs either so we all know better than submitting a bunch of AI slop for review).

I'd reject these PRs outright (wouldn't even review them), then arrange talks with the developers producing them to educate them on the dangers of submitting AI generated crap.

You're a developer, you write the code and use AI as a tool like any other. It shouldn't be used to replace people, it's not good enough and can actually be dangerous (as noted).

I think most of the problem here is about educating properly. Also raise this with your manager (if they're any good / will do anything about it).

Show threadAlrecenkMay 24
@Lucomo I encountered a website recently where a two factor authentication window was a popup, and I was able to continue by clicking on the page behind it without putting in a code. I can't prove it was an AI generated website, but all the signs were there. Not gonna name the site because it's a company whose actual product is not software, and I like what they actually make. I contacted their support to let them know because I don't want them to destroy themselves by vibe coding gaping holes into their security.
Show threadKristof ProvostMay 24
@Lucomo The thing that scares me most here isn't the people using LLMs to generate PRs. There's always going to be lazy, incompetent developers, especially when management encourages it in a misguided effort to cut costs.
The scary thing is the people defending using LLMs, in the very topic explaining how it doesn't work. Or the people who think that more LLM is the answer here, rather than a blanket policy of "No LLM generated code at all".
Show threadTim Ward ⭐🇪🇺🔶 #FBPEMay 24

@Lucomo "endless redirection where multiple things don't actually do anything"

Let me guess ... a million to one on this was trained on EJB code.

Show threadslotosMay 24

@Lucomo @baldur The idea that _tests_ should be open for AI edits will be the downfall of the industry.

The whole „AI will replace developers” thing is a self fulfilling prophecy. Reduce engineering to bullshit code generation, and of course AI is better at it.

Show threadRosarevenMay 24
@Lucomo The more I read about AI slops the more I think of I, Robot the book (not the movie).
Show threadStumpy The MuttMay 24
@Lucomo Of course the embedded ad says "Don't waste time fixing code security issues. Let our AI agents fix them for you"
Show threadKevin Karhan May 24

@Lucomo the only valid solution I see is flatout #ban users for that shit citing it as #vandalism.

  • The University of Minnesota got yeeted from #Linux fir far less shite!
Show threadSydMay 24
@Lucomo thanks for sharing this "[...] no one wants to talk about it." !!! we should only talk about this. This is the thing AI can't do and this is what hurts the industry most.
Show threadembixMay 24
@Lucomo The problem already starts with "TL (dev/manager hybrid)", AI slop is only the icing on that effing cake.
Show threadfrandroid - Mahmoud Khalil is free!!May 24
Half of this is bad management though.
Show threadEckes May 25
@Lucomo those Junk PRs are from their team and not from the AI. maybe he should manage their team.. or at least the team agrees on a mode of working.
Show threadSimonMay 26
@Lucomo gotta hit those 5k line PRs with one of these.
Show threadBorisMay 27
@Lucomo @ivan why, solution is obvious… have AI review those PRs…
Show threadirina 🐇🌷May 30
@Lucomo ah yes, the Volkswagen approach to testing