Mastodawn

jonny (good kind)1d ago

It's so cool that anthropic is setting up a double-sided protection racket where it will profit from the massive token burn of attackers and defenders with a tool specifically designed to generate exploits and their only observable mitigation is a clientside system prompt that sternly warns the LLM to be good and not do malware
https://red.anthropic.com/2026/mythos-preview/

Claude Mythos Preview \ red.anthropic.com

jonny (good kind)1d ago

sure they are doing """alignment""" to the models, and maybe they have some more sophisticated serverside mitigations. but the fact that the system prompt text is in the package at all rather than all being entirely serverside does the opposite of inspire confidence. Even the system prompt is fine with hacking as long as you go "it's ok I am good"
https://neuromatch.social/@jonny/116325221458366596

jonny (good kind)1d ago

so this simultaneously raises the floor of doing open source at all to "if you can afford brute force generating exploits against your repos for days at a time" while simultaneously causing so many false positives that bug bounties are crumbling and the info giants will pull labor from open source projects by just generating them badly in-house - don't roll your own crypto becomes "now you have to roll your own crypto because nobody else is, and then pay an AI company to secure it for you."

The end of the curl bug-bounty

tldr: an attempt to reduce the terror reporting. There is no longer a curl bug-bounty program. It officially stops on January 31, 2026. After having had a few half-baked previous takes, in April 2019 we kicked off the first real curl bug-bounty with the help of Hackerone, and while it stumbled a bit at first … Continue reading The end of the curl bug-bounty →

daniel.haxx.se

Nic Junius 1d ago

jonny (good kind)

you know that problem where it's actually in Google's best interests to sabotage their traditional search results to force everyone to use the AI results because then you never leave the site and direct prompt advertising becomes extremely valuable? yeah, it's like that for code, where it's actually in anthropic's best interests for all the code to be entirely unmaintainable and unsecurable except for with LLMs

jonny (good kind)1d ago

i feel bad constantly fixating on the informational capitalists while there is so much material harm being done in the world, and then simultaneously remind myself that this is literally capitalism's gamble to finally and fully enclose not only the material world but also our minds. If informational reality comes to be owned by 4 megacorporations, then it's all fascism forever baby.

happyborg 1d ago

@jonny I don't feel bad for that because the best thing I can do is use my understanding and skills. I know what I'm rubbish at too.

What I don't understand is why so few of those I came to know in the FOSS, privacy and security space do not get this, and are raving about it actively embracing "AI" and the snake oil hype.

Neil Moffatt 1d ago

@jonny All while Marx is still attacked as some kind of evil imposter ... mostly because he foretold the decay of humanity capitalism would herald.

You can enable a lot of evil by capturing and controlling information.

In many cases, the informational evil precedes and facilitates the actual harms. See: JK Rowling's TERF campaign and how genocides tend to work (you can't just kill a bunch of people out of the blue you have to get ordinary people to hate or at the very least not care for them first)

Lord Caramac the Clueless, KSC 1d ago

@jonny Not forever, only until the end of the Industrial Age, which won't last for very much longer, maybe another 50 to 70 years if things keep getting worse.

@LordCaramac @jonny the issue is they'll take a solid chunk of the biosphere down with them

Lord Caramac the Clueless, KSC 1d ago

@yakmacker @jonny We will probably lose at least half of all species of plants and animals. If we can keep the 6th Extinction within that range, we will most likely survive as a species even though our numbers will dwindle. However, if we let the ongoing extinction event escalate to the point where 85-95% of all species go extinct, we won't make it.

@LordCaramac @jonny I dunno if this kind of reasoning is really worth doing. A lot of the viability of humans depends on not just the percent of species that persist, but also which species persist, and I don't think anyone could provide a comprehensive list.

Another issue is that this appears to be a very gradual and non-synchronous mass extinction event, making the long term effects even harder to predict. The K-PG event, for example, probably started with a single apocalyptic event that made the entire world uninhabitable for large animals overnight. It's pretty unclear based on my casual reading on whether that is true of other famous extinction events like the P-T event.

unable to decrypt message 19h ago

@jonny Also, the same bullshit generators are being used to construct a facade for those doing material harm to pretend to avoid accountability. They're using these machines to move faster and break more things and people. It's all deeply connected. https://techwontsave.us/episode/322_why_iran_is_attacking_data_centers_w_sam_biddle

Why Iran is Attacking Data Centers w/ Sam Biddle - Tech Won’t Save Us

Tech Won't Save Us

@jonny the cyberpunk wiki is starting to look like the necronomicon now

jonny (good kind)1d ago

jonny (good kind)1d ago

@Viss i can't wait for the phase of the grift where "they can't control it" and release a series of whitepapers on how the only mitigation is to constantly refactor your code with a background churn of 10 exploit generation agents to not present a stable attack surface

jonny (good kind)1d ago

@Viss like their entire corporate voice is laying the groundwork for one day claiming "hey everyone now that we are too big to fail and integrated everywhere, we are unhappy to announce that we have lost control of the models but can't shut them off because their so important and everyone needs to subscribe to our active countermeasures protection suite or a rogue AI that we are no longer responsible for will hack you."

The Orange Theme 1d ago

@jonny @Viss Meanwhile, I'm in every vibe-coded web app going "../../../../../" popping 0-days lol.

@theorangetheme @jonny have you two seen the mythos stuff outta anthropic now?

jonny (good kind)21h ago

@Viss
@theorangetheme
In top of thread, or do you mean even more now than that?

@jonny @theorangetheme oh, shit. sorry. ignore me. ive had one fucker of a day. forgot this thread started with the mythos thing

jonny (good kind)21h ago

@Viss
@theorangetheme
No prob. I am not a security person so I can't really evaluate any of the claims in it, all I know is a protection racket when I see one

@jonny @theorangetheme oh this is 100% this scene: https://www.youtube.com/watch?v=NwYnbwQ0kv4

"Live Free, Don't Join" film clip - The Last Jedi Episode VIII Star Wars

I really like this scene and wanted to share it with the fabulous folks of the galaxy.OBVIOUSLY I DO NOT OWN THIS.Thanks Disney/Lucasfilm for letting me post.

YouTube

The Orange Theme 21h ago

@Viss @jonny I'm sorry you've had a bad day. I hope tomorrow is better.

@theorangetheme @jonny thanks, yeah me too.

jonny (good kind)1d ago

(for the literal reader out there, i am not claiming this is actually their secret plan or whatever, i am saying that whenever anthropic goes like "we didn't fully understand the model..." or invoke emergence or otherwise write as if the model is some unknowable god, that's always in service of product

Kusuriya (kk7hut)21h ago

@jonny @Viss I honestly see something like the events that lead to the black wall in Cyberpunk 2027 or the turing agents in the necromancer where we get that rogue AI and someone finally does something half assed trying to put the magic smoke back.

@kusuriya @jonny we're safe til they find flaws in tcp itself. til then, firewalls will still work

the only truly secure software is software that won't run, which "ai" is actually pretty good at 🤔

RootWyrm 🇺🇦

@Viss @jonny Claude barado nikto.

@rootwyrm @jonny oh shit thats a good one

João Santos 1d ago

@jonny I was going to say LLMs also struggle to understand LLM generated code, but then I remembered that it will only lead to more tokens being used, and increase Anthropic’s revenue, so it's a win-win for them.

jonny (good kind)1d ago

@jmcs exactly

cynicalsecurity

@jonny which is why I am really uncomfortable with their "vulnerability hunting" partnership which looks a lot like a closed club where "responsible disclosure" could well mask "no disclosure" or "only to the right US people disclosure"...

Claus Cramon Houmann 1d ago

@cynicalsecurity @jonny yes!

Joshua Barretto 1d ago

@jonny I've noticed a thing happening a lot recently where I build something, someone asks how it works, and is shocked at how little code there is behind the scenes. Maybe this isn't distinctly a thing that follows from LLMs alone, but it feels like people increasingly expect complexity where little is warranted, and that can't be a good omen

jonny (good kind)1d ago

@jsbarretto my friend i am watching this happen in real time too as projects that should take 1k lines suddenly take 10k lines of disjointed nonsense, and it's like "no it's not magic it's just thinking about how things should work a little bit"

@jonny
This feels really insightful, I gotta say!

Chris [list of emoji]1d ago

"First one's free."

synlogic4242 1d ago

@jonny bingo. like the scams where 2 guys enter a "mark" town. 1 begins breaking windows at night around town. by day the other guy goes door to door asking shopkeeps to pay him a protection fee. those who do miraculously cease having the broken windows happen