Serge from BabkaSep 4, 2025

After putting out a job posting the other day, nearly all the applications have LLMs used somewhere, whether that be the cover letters, their resume, or most concerning LLM written code on their github.

I have no problem using LLMs to help one's writing- I've done it myself, and I have no problem using LLM generated code when done thoughtfully and heavily supervised. I've done that too, but what I'm seeing are multiple candidates putting up a github profile and then using LLMs to generate sample projects they're passing off as their own.

This shifts things from "computer aided" to outright fraud. By analogy, it would be like sending someone your art portfolio and it's a bunch of AI generated artwork.

One could make the argument that this is good prompt engineering, but they're passing it off as their own work.

#LLM #AI #jobs

Show threadSerge from BabkaSep 4, 2025

These candidates are often passing themselves off as Open Source software developers or maintainers, and very easy to determine if that's true by looking at their forge profiles.

The criteria I use ranges from very easy to fake to nearly impossible.

1. Are the only projects they have the ones they've written, or do they have other projects as well?
2. For their own projects, do they have a history? Most that I'm seeing are a single commit, or a single day of commits, and then nothing. Are they maintained over time?
3. Do they have a history of other projects, such as writing issues, making patches, etc. to external projects that I've heard of/have prominence?
4. Does their own code "smell LLMy". There are telltale signs of LLM codebases. I won't go into them here.
5. If they do have commits over a period, how do those commits look? Do they fit the kind of commits I'd want in a workplace?

You might be able to fake some of these (and I've seen a few try) but not all.

Show threadCristina DeLisle
@serge I once saw a candidate whose GitHub was full of forks and nothing else. It looked like a very active profile with lots of projects, but in fact they hadn’t written a single line of code themselves - whether with LLMs or on their own.
Sep 4, 2025 at 1:00pmWeb
Show threadSerge from BabkaSep 4, 2025

@redchrision

I've seen that a little, but it seems like that's easy to spot by looking at their activity, looking at actions they've taken rather than just looking at the projects they have.