So Anthropic employees are using Claude Code to contribute AI-generated code to open source repositories and hiding the fact using their own internal “undercover mode”.
Totally trustworthy people.
(Any open source project that at the very least requires disclosure of AI-authored contributions should immediately ban Anthropic employees on principle.)
@aral Honestly I don't actually hate this.
It's a tool. The _user_ is responsible for what they're submitting. It's putting code generated by them in their name. I think this is actually good.
@glyph Yeah, I disagree. Code isn't ingredients and it's not “contamination" any more than you should label “I used search and replace on this”
What you want to know is whether it was well engineered or not.
And in fact, this is almost entirely orthogonal to "safety”. This is an engineering product. The safety comes from processes and whether or not _anyone checked the work done was right_, not the inputs.
@aredridel @glyph What "AI coding tools" *should* be putting in commit messages is:
Co-Authored-By: An unknown and unknowable set of people who did not consent to their work being used this way and to which there is no license for inclusion.
@dalias Morally arguable but not actually true under the copyright regime that exists.
At what point does learning from others constitute their authorship?
@aredridel @dalias it is true.
And LLMs cannot learn. They are merely a lossy compression/decompression thing. They regurgitate a somewhat averaged completion of the prompt from the other works they ingested.
@mirabilos
Not, strictly, true, though I get what you're going at.
There's a few phenomena going on that shape these tools beyond that.
- Emergent complexity
- "Memory" records
- Embedded context
- Incorporating social inputs
I still think they are strictly tools, but ones that can self-adapt with alarming power if you configure them right.
They have a medianizing effect on a lot of their output (that's actually one of the reasons they're good at code. We generally want code to be "normal". It's one of the many reasons it's pretty bad for more artistic creative work, morally and technically.)
But that's not the same as only repeating the median. The temperature, the randomness injected in makes them actually jump to stuff that is at times nonsensical but also at times clever. It's just randomness, but then with a heap of context and congruence applied that is rather interesting.
Aral Balkan
Mx. Aria Stewart
Glyph
Cassandrich
mirabilos