What's anthropic going to do, sue them? Insist in court that LLM recreating copyrighted code is a violation of copyright???
jonny (good kind)_I_VERIFIED_THIS_IS_NOT_CODE_OR_FILEPATHS and the docstring explains this is "to confirm you've verified the message contains no sensitive data." Like the LLM resorts to naming its variables with prompt text to remind it to not leak data while writing its code, which, of course, it ignores and prints the error directly.
So the reason that Claude code is capable of outputting valid json is because if the prompt text suggests it should be JSON then it enters a special loop in the main query engine that just validates it against JSON schema (it looks like the schema just validates that something in fact and object and its keys are strings) and then feeds the data with the error message back into itself until it is valid JSON or a retry limit is reached.
This code is so eye wateringly spaghetti so I am still trying to see if this is true, but this seems to be how it not only returns json to the user, but how it handles all LLM-to-JSON, including internal output from its tools. There appears to be an unconditional hook where if the JSON output tool is present in the session config at all, then all tool calls must be followed by the "force into JSON" loop.
If that's true, that's just mind blowingly expensive
edit: please note that unless I say otherwise all evaluations here are just from my skimming through the code on my phone and have not been validated in any way that should cause you to be upset with me for impugning the good name of anthropic
edit2: this is both much worse and not as bad as i thought on first read - https://neuromatch.social/@jonny/116326861737478342
Attached: 3 images OK i can't focus on work and keep looking at this repo. So after every "subagent" runs, claude code creates *another* "agent" to check on whether the first "agent" did the thing it was supposed to. I don't know about you but i smell a bit of a problem, if you can't trust whether one "agent" with a very big fancy model did something, how in the fuck are you supposed to trust another "agent" running on the smallest crappiest model? That's not the funny part, that's obvious and fundamental to the entire show here. HOWEVER RECALL [the above JSON Schema Verification thing](https://neuromatch.social/@jonny/116325123136895805) that is unconditionally added onto the end of every round of LLM calls. the mechanism for adding that hook is... JUST FUCKING ASKING THE MODEL TO CALL THAT TOOL. second pic is registering a hook s.t. "after some stop state happens, if there isn't a message indicating that we have successfully called the JSON validation thing, prompt the model saying "you must call the json validation thing" this shit sucks so bad they can't even ***CALL THEIR OWN CODE FROM INSIDE THEIR OWN CODE.*** Look at the comment on pic 3 - "e.g. agent finished without calling structured output tool" - that's common enough that they have a whole goddamn error category for it, and the way it's handled is by just pretending the job was cancelled and nothing happened.
USER_TYPE=ant that seems to be read directly off the node env var accessor. No idea how much of that would be serverside verified but boy is that sloppy. They are often labeled in comments as "anthropic only" or "internal only," so the intention to gate from external users is clear lol
reminder that anthropic ran (and is still running) an ENTIRE AD CAMPAIGN around "Claude code is written with claude code" and after the source was leaked that has got to be the funniest self-own in the history of advertising because OH BOY IT SHOWS.
it's hard to get across in microblogging format just how big of a dumpster fire this thing is, because what it "looks like" is "everything is done a dozen times in a dozen different ways, and everything is just sort of jammed in anywhere. to the degree there is any kind of coherent structure like 'tools' and 'agents' and whatnot, it's entirely undercut by how the entire rest of the code might have written in some special condition that completely changes how any such thing might work." I have read a lot of unrefined, straight from the LLM code, and Claude code is a masterclass in exactly what you get when you do that - an incomprehensible mess.
from @sushee over here, (can't attach images in quotes) and apparently discussed on HN so i'm late, but...
They REALLY ARE using REGEX to detect if a prompt is negative emotion. dogs you are LITERALLY RIDING ON A LANGUAGE MODEL what are you even DOING
OK i can't focus on work and keep looking at this repo.
So after every "subagent" runs, claude code creates another "agent" to check on whether the first "agent" did the thing it was supposed to. I don't know about you but i smell a bit of a problem, if you can't trust whether one "agent" with a very big fancy model did something, how in the fuck are you supposed to trust another "agent" running on the smallest crappiest model?
That's not the funny part, that's obvious and fundamental to the entire show here. HOWEVER RECALL the above JSON Schema Verification thing that is unconditionally added onto the end of every round of LLM calls. the mechanism for adding that hook is... JUST FUCKING ASKING THE MODEL TO CALL THAT TOOL. second pic is registering a hook s.t. "after some stop state happens, if there isn't a message indicating that we have successfully called the JSON validation thing, prompt the model saying "you must call the json validation thing"
this shit sucks so bad they can't even CALL THEIR OWN CODE FROM INSIDE THEIR OWN CODE.
Look at the comment on pic 3 - "e.g. agent finished without calling structured output tool" - that's common enough that they have a whole goddamn error category for it, and the way it's handled is by just pretending the job was cancelled and nothing happened.
@jonny I’m fascinated by the breakdown you’re having today, and, trust me, this is commiseration not me being patronizing.
But…are people just now realizing that these models are just a human centipede? Which is to say “a bunch of models sutured together, ass-to-mouth, one feeding excrement to the next?”
Because, I mean, that’s all they are. A human centipede of LLMs constructed by overpaid morons.
The finest minds from America’s elite educational institutions
@jonny "don't show error to user"
This shit really keeps on giving.
@jonny their velocity for shipping *slop* is indeed insane
😜
@jonny "velocity for shipping is insane"
it turns out you can ship very fast if nothing has to work!
@whitequark @jonny Apparently some have had DMCA takedowns filed against them, so here are a couple links still working as of this writing:
@whitequark @jonny Additionally:
Claude Code is an agentic coding tool that lives in your terminal, understands your codebase, and helps you code faster by executing routine tasks, explaining complex code, and handling git workflo...
Paul Rietschka
The Orange Theme
GhostOnTheHalfShell
G. Clavier
pibert π
Cap Ybarra
mmby
Alexander The 1st
𝔸𝕟𝕔𝕚𝕖𝕟𝕥 𝕊𝕠𝕦𝕟𝕕𝕤 🔉
Matthew Lyon
Knud Jahnke
Paco Hope
⁂ L. Rhodes
Lunar 🛸 ♾
Edwin Groothuis
Jason Bowen 🇺🇦
Blake C. Stacey
damien 🥖🐈⬛🧣
Matt Hall
Josh Sutphin
Jared White (ResistanceNet ✊)
Reid 
catch
Andrew
beemoh
Vlad 🇺🇦
Tom Casavant
✧✦Catherine✦✧
Jamie Gaskins
Toast, Anonymous Fedi Fungus
nash
Brodeuse LucileDT
Dan Sugalski
Preston Maness ☭
[HANDMAIDEN] xan
Lars Marowsky-Brée 😷
José Albornoz