SnoopJ

Bruce Schneier was on today's edition of The Tech Report, talking about the newest wank out of Anthropic.

He made a claim that he'd seen researchers compare the asserted performance of the new model (in the "system card" etc.) against previously-available models, and they were performing at about the same level (finding the same vulns, it sounded like).

Does anybody know what research he was referring to?

EDIT: FOUND! https://aisle.com/blog/ai-cybersecurity-after-mythos-the-jagged-frontier

A web search is not pulling anything up for me, because the infosphere is currently flooded with credulous repetition of Anthropic's marketing materials (and I have no reason to believe those aren't just lies)

AI Cybersecurity After Mythos: The Jagged Frontier

Why the moat is the system, not the model

AISLE
Apr 9 at 6:09pmWeb
Show threadDaniel Gibson1d ago
@SnoopJ
vaguely related: https://mastodon.social/@bagder/116362046377975050
Show threadSnoopJ1d ago

@Doomed_Daniel this raises an interesting question that I don't see represented in the current round of breathless hype:

I wonder if the vuln-finding capabilities are not really much better, but the generation of quality reports is?

I'm sure it'll be clear when the dust settles in a month or two, in any event.

Show threadDaniel Gibson1d ago
@SnoopJ
I guess the most important factor would be fewer false positives - if the AI reports 100 vulns and only 5 of them are valid it's not overly useful
Show threadSnoopJ1d ago
@Doomed_Daniel yea this is something that came up in the discussion on TTR
Show threadTim Farley1d ago
@SnoopJ I found the report that Schneier was talking about on the AISLE blog here: https://aisle.com/blog/ai-cybersecurity-after-mythos-the-jagged-frontier
AI Cybersecurity After Mythos: The Jagged Frontier

Why the moat is the system, not the model

AISLE
Show threadSnoopJ1d ago
@krelnik thank you so much! my search skills were failing me but I figured Mastodon might come to my rescue, and indeed it did