sweisAssessing Claude Mythos Preview's cybersecurity capabilities
A very good outcome for AI safety would be if when improved models get released, malicious actors use them to break society in very visible ways. Looks like we're getting close to that world.
Gives me Fight Club vibes.
I'd love to see them point at a target that's not a decades old C/C++ codebase. Of the targets, only browsers are what should be considered hardened, and their biggest lever is sandboxing, which requires a lot of chained exploits to bypass - we're seeing that LLMs are fast to discover bugs, which means they can chain more easily. But bug density in these code bases is known to be extremely high - especially the underlying operating systems, which are always the weak link for sandbox escapes.
I'd love to see them go for a wasm interpreter escape, or a Firecracker escape, etc. They say that these aren't just "stack-smashing" but it's not like heap spray is a novel technique lol
> It autonomously obtained local privilege escalation exploits on Linux and other operating systems by exploiting subtle race conditions and KASLR-bypasses.
I think this sounds more impressive than it is, for example. KASLR has a terrible history for preventing an LPE, and LPE in Linux is incredibly common. Has anything changed here? I don't pay much attention but KASLR was considered basically useless for preventing LPE a few years ago.
> Because these codebases are so frequently audited, almost all trivial bugs have been found and patched. What’s left is, almost by definition, the kind of bug that is challenging to find. This makes finding these bugs a good test of capabilities.
This just isn't true. Humans find new bugs in all of this software constantly.
It's all very impressive that an agent can do this stuff, to be clear, but I guess I see this as an obvious implication of "agents can explore program states very well".
edit: To be clear, I stopped about 30% of the way through. Take that as you will.
> Mythos Preview identified a memory-corruption vulnerability in a production memory-safe VMM. This vulnerability has not been patched, so we neither name the project nor discuss details of the exploit.
Good morning Sir.
> Has anything changed here? I don't pay much attention but KASLR was considered basically useless for preventing LPE a few years ago.
No. It's still like this. Bonus point that there are always free KASLR leaks (prefetch side-channels).
But then, this thing is just.. I don't have a word for this. Just randomly read paragraphs from the post and it's like, what?
Oh, that. That's true, I didn't know Mythos found that one. I guess I will not comment further on it until there's a write up (edited out a bit more).
> It is easy to turn this into a denial-of-service attack on the host, and conceivably could be used as part of an exploit chain.
So yeah, perhaps some evidence to what I'm getting at. Bug density is too low in that project, it's high enough in others. I'll be way way way more interested in that.
> But then, this thing is just.. I don't have a word for this. Just randomly read paragraphs from the post and it's like, what?
I read about 30% and got bored. I suppose I should have been clearer, but my impression was pretty quickly "cool" and "not worth reading today".
> I read about 30% and got bored.
I was lucky then :) Somehow I saw this first. And then the "somewhat reliably writing exploits for SpiderMonkey" part, and then the crypto libraries part. Finally I wonder why is there a Linux LPE mini writeup and realized it's the "automatically turn a syzkaller report to a working exploit" part.
Now that I read the first few things (meh bugs in OpenBSD, FFmpeg, FreeBSD etc) they are indeed all pretty boring!
If people want exploitable syzkaller reports, following spender is free!
The majority of vulnerabilities are in newly committed lines of code. This has been shown again and again [1] [2]
From a marketing standpoint Anthropic is showing that they're able to direct 'compute' to find vulnerabilities where human time/cost is not efficient or effective.
Project Glasswing is attempting to pay off as many of these old vulnerabilities as possible now so the low-hanging fruit has already been picked.
The next generation of Mythos and real world vulnerabilities exploits are going to be in newly committed code...
> The majority of vulnerabilities are in newly committed lines of code. This has been shown again and again
That's fine, I wouldn't argue against that. It doesn't really change things, right?
> From a marketing standpoint Anthropic is showing that they're able to direct 'compute' to find vulnerabilities where human time/cost is not efficient or effective.
Yes, they've demonstrated that.
Related ongoing threads:
System Card: Claude Mythos Preview [pdf] - https://news.ycombinator.com/item?id=47679258
Project Glasswing: Securing critical software for the AI era - https://news.ycombinator.com/item?id=47679121
I can't tell which of the current threads, if any, should be merged - they all seem significant. Anyone?
There is a lot to digest here. Maybe having a few separate pages makes them a bit more digestible. The system card itself is some 200 odd pages
The elephant in the room here is that there are hundreds of millions of embedded devices that cannot be upgraded easily and will be running vulnerable binaries essentially forever. This was a problem before of course, but the ease of chaining vulnerabilities takes the issue to a new level.
The only practical defense is for these frontier models to generate _beneficial_ attacks to innoculate older binaries by remote exploits. I dubbed these 'antibotty' networks in a speculative paper last year, but never thought things would move this fast! https://anil.recoil.org/papers/2025-internet-ecology.pdf
No, the elephant in the room is that even bad actors will now have easier to find vulnerabilities in, maintained or not, widely or in critical places used software. Unmaintained and remotely accessible devices should be discarded as soon as possible, you can't stay waiting till some of the good guys decide to give some time to your niche but critical unmaintained piece of software. Because if there is a possibility of taking profit of it, it will be checked and exploited.
And you can't assume that whatever vulnerability they have will let good guys to do the extra (and legally risky) work of closing the hole.
Since this level of security ”scanning” requires heaps of money, this is going to kill off a substantial part of F/OSS.