So Anthropic employees are using Claude Code to contribute AI-generated code to open source repositories and hiding the fact using their own internal “undercover mode”.
Totally trustworthy people.
(Any open source project that at the very least requires disclosure of AI-authored contributions should immediately ban Anthropic employees on principle.)
@aral Honestly I don't actually hate this.
It's a tool. The _user_ is responsible for what they're submitting. It's putting code generated by them in their name. I think this is actually good.
@glyph Yeah, I disagree. Code isn't ingredients and it's not “contamination" any more than you should label “I used search and replace on this”
What you want to know is whether it was well engineered or not.
And in fact, this is almost entirely orthogonal to "safety”. This is an engineering product. The safety comes from processes and whether or not _anyone checked the work done was right_, not the inputs.
@aredridel @glyph What "AI coding tools" *should* be putting in commit messages is:
Co-Authored-By: An unknown and unknowable set of people who did not consent to their work being used this way and to which there is no license for inclusion.
@dalias Morally arguable but not actually true under the copyright regime that exists.
At what point does learning from others constitute their authorship?
@aredridel LLM slop is nothing like "learning from others".
But if you recall, we even took precautions against that. FOSS projects reimplementing proprietary things were careful to exclude anyone who might had read the proprietary source, disassembled proprietary code, worked at the companies who wrote or had access to that code, etc.
@dalias @aredridel A test which LLMs fail by the very virtue of their functioning mechanisms.
It's all fundamentally derivative of the training dataset and it has been exposed both to AGPL and to proprietary datasets.
@lispi314 If you're making claims about copyright law — like whether something is derivative — and to legal documents like the AGPL, legal authority is very much relevant.
Programmers really gotta stop treating licenses like they're code that get executed. That's not how it works. That's not how any of this works.
legal authority is very much relevant.
The court of today is not the court of tomorrow. Therefore you do not take any risk on the matter if you want to be absolutely safe (you do, especially if your code is infrastructure of any sort and has to be valid everywhere).
That being said, I am also taking the argument from an ideological stance.
One does not simply include Proprietary Malware into Free Software.
It is disrespectful of the users' Freedoms and to oneself.
And in the case of plagiarized Free Software, it is still disrespectful to not to provide due reference to the source's original author.
@aredridel @lispi314 The facts of the matter are completely and utterly obvious.
Now, we live in a world where legal authorities are under complete capture by billionaires pushing this drug, so I am not going to make any predictions about how courts will rule. Even if they do rule in favor of these companies, those rulings will not be treated as precedents that benefit us.
And they will not be accepted by our communities.
What defines FOSS is not whether a court says it's non-infringing, but whether our communities agree that it was made respecting the intent and consent of the authors who licensed it.
@dalias Have you checked with the Free Software Foundation about that?
(Seriously, if it's a moral argument you're making, it's way stronger if you actually make it!)
Now "respect the intent of the author" is a fascinating concept and one worth examining!
@dalias Right. You're appealing to a definition of "FOSS" that isn't entirely clear what it is. And the people who do usually have (some) claim to that authority, the common uses of it, are not the ones you're using.
I'm sympathetic to that but I can't tell what it is in an appeal to an unstated norm for a community that I can't quite identify.
@aredridel @dalias it's an existing community that's pretty well-defined as:
Everyone who believes that the *intent* of open-source licenses should be respected regardless of whether legal machinations actually enforce that.
It's super interesting to observe right now how that community is smaller than "people who say they're committed to FOSS" but the community is clearly at least a substantial subset of open-source contributors and maintainers, and regardless of what happens with the whole current "AI" debacle, we're mostly going to continue building human-authored code, giving it away for free (with an attribution requirement or more) and hoping that others will respect that simple requirement, and shaming/shunning those who flaunt it and brag about doing so (or in this case try as hard as they can to maliciously break the good citation practices and attributions that are part of the lifeblood of the community.
Some think this community will shrink and atrophy over time; others imagine it will be around cleaning up the mess after the AI bubble bursts. Whatever your expectation, saying "I think it's fine for Anthropic employees to actively undermine open-source attribution principles" tells everyone clearly that you're not interested in being part of the community that cares about those.
@tiotasram @dalias Yeah, that's never been at all unified. As long as I've participated in free software and free culture movements, there's always been a legalist side, an ideological side, and a community oriented side at least, plus the schism of 'permissive' vs 'copyleft'.
Never mind the corporate vs hacker aspects.
There's certainly a ton of different ideological approaches that contradict each other coming into play; IMO that makes for a healthy community from the anarchist perspective, and events like this where incompatible parts of it shuffle off are normal and acceptable. I'm going to vigorously oppose LLM-generated code and those who defend/promote it, the people in that camp who once believed themselves to be part of the "open source community" are going to have to recon with the fact sooner or later that the tools they promote are inimical to that community, and one side or the other will win the ideological battle over the term "open source" but the two camps won't be collaborating as freely any more given that one of them is actively preying upon the work of the other.
So far to me this schism seems much deeper than the permissive vs. copyleft debate (although to some extent it cuts along some of the same fault lines).
@bms48 Yep. Not the model I want to see, but one I think is quite justified (and probably the right one in a few cases)
I want to see more people look at making software _designed_ in its structure to be open and liberatory and not be used in corporate control systems so much. It means making things that are shaped differently: less oauth2 and free libraries to implement common control structures, and more like systems built to give users direct control to manipulate their structure. Plugins and primitives that can be combined and reordered and self-hosting and peer to peer.
Like, the license has never particularly kept things being very good, and the stuff that _really_ gets widely used is permissively licensed anyway.
I've always thought that _what_ we are building should make it undesirable to copy and use in a corporate setting.
@bms48 Really good case in point!
Most of my stuff is also permissively licensed, though I've experimented with other models a bit. I think in the end I've reaped quite a bit of reward, but none of it directly financial. Lots of "clout" I guess you'd call it, and a fair number of jobs where the amount of open work I've done sure factored into it. So in many ways, similar, if less definite.
@aredridel @dalias
> but not actually true under the copyright regime that exists
Under the copyright regime that exists in the US specifically, the generated code is at best not copyrightable at all (and therefore cannot be included into any projects with licenses relying on copyright).
Of course maintainers of said projects might decide to yolo it, but also they might decide to not; and in this case, the intentional deception by antropic becomes even more significant fraud.
@aredridel @dalias but in this specific case, the people submitting PRs are the one that created the tool.
We're talking specifically about a tool developed by Anthropic, which has a separate mode for Anthropic employees, which purposefully is "operating undercover" and creating MRs that mislead OSS maintainers about provenance of these MRs.
@IngaLovinde Are there any examples of misleading out there?
Or is it _just not mentioning it_?
@aredridel when a developer submits code, the default assumption is that they wrote it (and not, say, plagiarized it from somewhere without actually understanding it).
And on the original screenshot, it's clear that not only do they not mention the actual provenance themselves, but that they go extra mile to ensure that it doesn't leak in any other way.
"You are operating UNDERCOVER", "do not blow your cover", "NEVER include [...] any [...] attribution" communicates intent very clearly and is a very clear admission of guilt, regardless of whether these magic instructions to LLM actually work or not.
@dalias @IngaLovinde @aredridel AFAICT it merely confirms that AI output cannot be copyrighted as new work of its own (naturally, as the human creativity aspect is missing and it’s merely an algorithmic transformation on a deterministic machine, PRNG inputs notwithstanding).
It does not reduce the claims of the authors of the works it ingested to regurgitate the output.
@dalias @mirabilos @aredridel which still means that at best the generated code cannot be copyrighted, and at worst it violated copyright and license terms of the original authors (whose works were ingested to train the model). In both these cases, the resulting code cannot be incorporated into any FOSS project with any license.
Typically when people submit code to FOSS projects, they also (implicitly or explicitly) claim that they hold the copyright on the submitted code, and agree that this code will be licensed under the license the project uses (which they only have power to do if the first claim is actually true).
When LLMs are used to generate code, the first claim is false, and it _is_ a contamination.
@aredridel How similar is it? Are there appreciable portions that are exactly the same? If so, the default assumption if they've *memorized* and *already proven themselves to have memoried* the thing they allegedly plagiarized is that it's plagiarism. There have been plenty of court cases over this in literature, in music, etc. It's not some vague unknown.
If they had never seen the original or maybe only saw (or for music, heart) it in passing, there might be more leeway for doubt.
Part of the consequences of having spent so much time looking at a work that you've memorized it is that you lose the ability to make things of your own that are similar to it but meaningfully "your own".
@aredridel @dalias people are still humans, not machines.
Are you a TESCREAList?
“the generated code is at best not copyrightable at all (and therefore cannot be included into any projects with licenses relying on copyright)”
Why would public domain code not be acceptable for inclusion into open source projects?
@IngaLovinde I agree that the licenses cannot apply to works without copyright, but why is that a problem? There is no requirement for the whole project to be under one single license, as long as they are all compatible.
I’m sure there are open source projects that ship a copy of SQLite, a well known public domain project.
Linux is considered a GPL project but there is code in the tree under more permissive BSD-style licenses.
@aredridel @dalias it is true.
And LLMs cannot learn. They are merely a lossy compression/decompression thing. They regurgitate a somewhat averaged completion of the prompt from the other works they ingested.
Aral Balkan
Mx. Aria Stewart
Glyph
Cassandrich
LisPi
Tiota Sram
Bruce Simpson, Ph.D.
mirabilos
Inga stands with 🇺🇦 🇵🇸
mxey