Mastodawn

jonny (good kind)9h ago

It's so cool that anthropic is setting up a double-sided protection racket where it will profit from the massive token burn of attackers and defenders with a tool specifically designed to generate exploits and their only observable mitigation is a clientside system prompt that sternly warns the LLM to be good and not do malware
https://red.anthropic.com/2026/mythos-preview/

Claude Mythos Preview \ red.anthropic.com

jonny (good kind)9h ago

sure they are doing """alignment""" to the models, and maybe they have some more sophisticated serverside mitigations. but the fact that the system prompt text is in the package at all rather than all being entirely serverside does the opposite of inspire confidence. Even the system prompt is fine with hacking as long as you go "it's ok I am good"
https://neuromatch.social/@jonny/116325221458366596

jonny (good kind)8h ago

so this simultaneously raises the floor of doing open source at all to "if you can afford brute force generating exploits against your repos for days at a time" while simultaneously causing so many false positives that bug bounties are crumbling and the info giants will pull labor from open source projects by just generating them badly in-house - don't roll your own crypto becomes "now you have to roll your own crypto because nobody else is, and then pay an AI company to secure it for you."

The end of the curl bug-bounty

tldr: an attempt to reduce the terror reporting. There is no longer a curl bug-bounty program. It officially stops on January 31, 2026. After having had a few half-baked previous takes, in April 2019 we kicked off the first real curl bug-bounty with the help of Hackerone, and while it stumbled a bit at first … Continue reading The end of the curl bug-bounty →

daniel.haxx.se

jonny (good kind)

you know that problem where it's actually in Google's best interests to sabotage their traditional search results to force everyone to use the AI results because then you never leave the site and direct prompt advertising becomes extremely valuable? yeah, it's like that for code, where it's actually in anthropic's best interests for all the code to be entirely unmaintainable and unsecurable except for with LLMs

jonny (good kind)7h ago

i feel bad constantly fixating on the informational capitalists while there is so much material harm being done in the world, and then simultaneously remind myself that this is literally capitalism's gamble to finally and fully enclose not only the material world but also our minds. If informational reality comes to be owned by 4 megacorporations, then it's all fascism forever baby.

happyborg 7h ago

@jonny I don't feel bad for that because the best thing I can do is use my understanding and skills. I know what I'm rubbish at too.

What I don't understand is why so few of those I came to know in the FOSS, privacy and security space do not get this, and are raving about it actively embracing "AI" and the snake oil hype.

Neil Moffatt 6h ago

@jonny All while Marx is still attacked as some kind of evil imposter ... mostly because he foretold the decay of humanity capitalism would herald.

@jonny the cyberpunk wiki is starting to look like the necronomicon now

jonny (good kind)8h ago

jonny (good kind)7h ago

@Viss i can't wait for the phase of the grift where "they can't control it" and release a series of whitepapers on how the only mitigation is to constantly refactor your code with a background churn of 10 exploit generation agents to not present a stable attack surface

jonny (good kind)7h ago

@Viss like their entire corporate voice is laying the groundwork for one day claiming "hey everyone now that we are too big to fail and integrated everywhere, we are unhappy to announce that we have lost control of the models but can't shut them off because their so important and everyone needs to subscribe to our active countermeasures protection suite or a rogue AI that we are no longer responsible for will hack you."

The Orange Theme 7h ago

@jonny @Viss Meanwhile, I'm in every vibe-coded web app going "../../../../../" popping 0-days lol.

jonny (good kind)7h ago

(for the literal reader out there, i am not claiming this is actually their secret plan or whatever, i am saying that whenever anthropic goes like "we didn't fully understand the model..." or invoke emergence or otherwise write as if the model is some unknowable god, that's always in service of product

João Santos 8h ago

@jonny I was going to say LLMs also struggle to understand LLM generated code, but then I remembered that it will only lead to more tokens being used, and increase Anthropic’s revenue, so it's a win-win for them.

jonny (good kind)8h ago

@jmcs exactly

cynicalsecurity

@jonny which is why I am really uncomfortable with their "vulnerability hunting" partnership which looks a lot like a closed club where "responsible disclosure" could well mask "no disclosure" or "only to the right US people disclosure"...

Claus Cramon Houmann 7h ago

@cynicalsecurity @jonny yes!

Joshua Barretto 7h ago

@jonny I've noticed a thing happening a lot recently where I build something, someone asks how it works, and is shocked at how little code there is behind the scenes. Maybe this isn't distinctly a thing that follows from LLMs alone, but it feels like people increasingly expect complexity where little is warranted, and that can't be a good omen

jonny (good kind)7h ago

@jsbarretto my friend i am watching this happen in real time too as projects that should take 1k lines suddenly take 10k lines of disjointed nonsense, and it's like "no it's not magic it's just thinking about how things should work a little bit"

@jonny
This feels really insightful, I gotta say!

Chris [list of emoji]3h ago

"First one's free."