So Anthropic employees are using Claude Code to contribute AI-generated code to open source repositories and hiding the fact using their own internal “undercover mode”.

Totally trustworthy people.

(Any open source project that at the very least requires disclosure of AI-authored contributions should immediately ban Anthropic employees on principle.)

#AI #Anthropic #ClaudeCode #subterfuge

@aral Honestly I don't actually hate this.

It's a tool. The _user_ is responsible for what they're submitting. It's putting code generated by them in their name. I think this is actually good.

@aredridel @aral I really can’t agree with this, because it’s a question of accurate labeling not of “responsibility” or “authorship”. co-authored-by is perhaps the wrong method for labeling such things, but consider raw milk. ultimately, it is indeed the producer’s responsibility to ensure their product is free of contamination. but disclosure of its method of production is explicitly the kind of requirement that allows consumers of said product to make safe choices

@glyph Yeah, I disagree. Code isn't ingredients and it's not “contamination" any more than you should label “I used search and replace on this”

What you want to know is whether it was well engineered or not.

And in fact, this is almost entirely orthogonal to "safety”. This is an engineering product. The safety comes from processes and whether or not _anyone checked the work done was right_, not the inputs.

@aredridel @glyph It is ingredients. It's not search-and-replace. It's literally incorporating parts of an unknown set of almost-surely-copyrighted works, without license or attribution, into the submission the person is misrepresenting as their own.

@aredridel @glyph What "AI coding tools" *should* be putting in commit messages is:

Co-Authored-By: An unknown and unknowable set of people who did not consent to their work being used this way and to which there is no license for inclusion.

@dalias Morally arguable but not actually true under the copyright regime that exists.

At what point does learning from others constitute their authorship?

@aredridel @dalias
> but not actually true under the copyright regime that exists

Under the copyright regime that exists in the US specifically, the generated code is at best not copyrightable at all (and therefore cannot be included into any projects with licenses relying on copyright).
Of course maintainers of said projects might decide to yolo it, but also they might decide to not; and in this case, the intentional deception by antropic becomes even more significant fraud.

@IngaLovinde @aredridel The ruling you're talking about was a case about actually *generated* code, before "gen AI" was a real thing, not obviously-derivative transformations of a corpus.

@dalias @IngaLovinde @aredridel AFAICT it merely confirms that AI output cannot be copyrighted as new work of its own (naturally, as the human creativity aspect is missing and it’s merely an algorithmic transformation on a deterministic machine, PRNG inputs notwithstanding).

It does not reduce the claims of the authors of the works it ingested to regurgitate the output.

@dalias @mirabilos @aredridel which still means that at best the generated code cannot be copyrighted, and at worst it violated copyright and license terms of the original authors (whose works were ingested to train the model). In both these cases, the resulting code cannot be incorporated into any FOSS project with any license.

Typically when people submit code to FOSS projects, they also (implicitly or explicitly) claim that they hold the copyright on the submitted code, and agree that this code will be licensed under the license the project uses (which they only have power to do if the first claim is actually true).
When LLMs are used to generate code, the first claim is false, and it _is_ a contamination.

@dalias @IngaLovinde @aredridel if something cannot be copyrighted and no others’ rights apply, then it is in the public domain. For LLM output, which has been proven to vastly resemble existing code under copyright, that’s not the case.
@mirabilos @IngaLovinde @aredridel Indeed it's been demonstrated that you can "coax" LLM chatbots into emitting large parts of their training corpus nearly verbatim, so it's clear that the works from the corpus, with minor degrees of lossiness, are contained within the models. And when they output something very similar, it's ridiculous trying to argue that the output isn't derivative too.
@dalias Have you seen how people perform on similar tests?
@aredridel If a person went in a room and memorized an existing program, then came out and, asked to write a program to do the same thing, wrote down something that was nearly identical to what they'd just gone and memorized, I think any reasonable person would agree that it was plagiarism, and copyright infringement if they attempted to publish the result without having license to do so.
@dalias How about if they emit something analogous but not the same?

@aredridel How similar is it? Are there appreciable portions that are exactly the same? If so, the default assumption if they've *memorized* and *already proven themselves to have memoried* the thing they allegedly plagiarized is that it's plagiarism. There have been plenty of court cases over this in literature, in music, etc. It's not some vague unknown.

If they had never seen the original or maybe only saw (or for music, heart) it in passing, there might be more leeway for doubt.

Part of the consequences of having spent so much time looking at a work that you've memorized it is that you lose the ability to make things of your own that are similar to it but meaningfully "your own".

@dalias Right. but then _is that present in the output actually in question_?