Saagar JhaDec 30
So there’s this guy on GitHub that’s sending hundreds of PRs to completely unrelated open source projects to fix typos and stuff and I’m sure he’s using AI or something because half of them are unnecessary or wrong and when someone reviews the PR to point it out he just closes it
Show threadSaagar JhaDec 30
There are no obvious tells and some of them do actually fix real (if often superficial problems) so he is being taken seriously by most of them, it’s just that half of them get merged and half are closed as per above because they’re wrong.
icrazeDec 31
Show threadSaagar Jha
Except sudo, which has merged all 23 of his PRs without comment. I guess this means their code is either so bad that the LLM is finding all bunch of actual bugs, or that the maintainers aren’t reading his PRs closely. I’m not sure either makes me feel good about the project.
Dec 30 at 1:49pmWeb
Show threadMatt MassicotteDec 30
@saagar sudo merge pr
Show threadGreg GardnerDec 30
@saagar At least sudo isn't a critical piece of the security of almost every Unix-based system on the planet. Oh wait, it is.
Show threadpancake Dec 31
@saagar link?
Show threadicrazeDec 31
@pancake @saagar GitHub user AZero13
Show threadpancake Dec 31
@icraze @saagar yeah 23 prs merged in one week. I hope @millert reviewed them properly before merging, but to me the changes look safe, actually there are a couple of tricky ones that are fixing real memory safety bugs.
Show threadpancake Dec 31
@saagar llms can spot a large variety of bugs that static analyzers don’t catch, yeah sometimes also hallucinate some. But combined with asan, clang analyzer and valgrind they are a really powerful solution for code review and bug catching. So I wouldn’t say sudo’s code is bad, but software engineering is so hard that it’s easy to find ways to improve the quality over time. C coding practices evolved and that’s a techdebt we are all used to face every day
Show threadSaagar JhaJan 1
@pancake I think it is possible to use AIs effectively but I don’t think you can send 150 PRs a month without there being problems somewhere
Show threadpancake Jan 1
@saagar yeah i doubt that too, im also not happy with blind merging or review less prs
Show threadkrigDec 31
@saagar @algernon good thing it’s just some fringe project and not a foundational building block that all computing is built upon
Show threadThomas TouheyDec 31
@saagar The maintainer does seem to have commented/done review on some, but some of the PRs either fix security issues, or introduce some, finding out which would require some more experience with the codebase. Anyway, very scary stuff.
Show threadTechokamiDec 31
@saagar Two of them did have comments, though: https://github.com/sudo-project/sudo/pull/498 this one points out an issue that was then resolved by him. https://github.com/sudo-project/sudo/pull/490 THIS one, an actual conversation (albeit a short one) on potentially deprecating and removing SecureID support
parse_args.c never says "the -E option is not valid in edit mode" by AZero13 · Pull Request #498 · sudo-project/sudo

This is because we are checking the wrong variable.

GitHub
Show threadSaagar JhaJan 1
@techokami Yeah fair I didn’t actually click through all 23
Show threadJakobpunkt🔥Dec 31
@saagar @gnomon wait wat
Show threadBen ZaninDec 31
@jakobpunkt other folks in the thread pointed out some additional information that mitigates my worry a little bit, but it's not a great look at all.