surprisetalkMaking frontier cybersecurity capabilities available to defenders
Anakin: I'm going to save the world with my AI vulnerability scanner, Padme.
Padme: You're scanning for vulnerabilities so you can fix them, Anakin?
Anakin: ...
Padme: You're scanning for vulnerabilities so you can FIX THEM, right, Annie?
Definitely will be a fight against bad actors pulling bulk open source software projects, npm packages, etc and running this for their own 0 days.
I hope Anthropic can place alerts for their team to look for accounts with abnormal usage pre-emptively.
There's a lot of skepticism in the security world about whether AI agents can "think outside the box" enough to replicate or augment senior-level security engineers.
I don't yet have access to Claude Code Security, but I think that line of reasoning misses the point. Maybe even the real benefit.
Just like architectural thinking is still important when developing software with AI, creative security assessments will probably always be a key component of security evaluation.
But you don't need highly paid security engineers to tell you that you forgot to sanitize input, or you're using a vulnerable component, or to identify any of the myriad issues we currently use "dumb" scanners for.
My hope is that tools like this can help automate away the "busywork" of security. We'll see how well it really works.
as a pentester at a Fortune 500: I think you're on the mark with this assessment. Most of our findings (internally) are "best practices"-tier stuff (make sure to use TLS 1.2, cloud config findings from Wiz, occasionally the odd IDOR vuln in an API set, etc.) -- in a purely timeboxed scenario, I'd feel much more confident in an agent's ability to look at a complex system and identify all the 'best practices' kind of stuff vs a human being.
Security teams are expensive and deal with huge streams of data and events on the blue side: seems like human-in-the-loop AI systems are going to be much more effective, especially with the reasoning advances we've seen over the past year or so.
As a founder of an auditing firm, I definitely feel the heat of the competition when big LLM companies push products that not only compete with us an auditors but also with our own AI-based offerings (https://zkao.io/).
If I were to venture a guess, there's different world in which we might exist in the next 5-10 years.
In one of these futures, we, as auditors, seize to exist. If this is the future, then developers seize to exist too, and most people touching software seize to exist. My guess here is as good as any developer's guess on if their job will remain stable.
In another one of these futures, us auditors become more specialized, more niche, and bring the "human touch" needed or required. Serious companies will want to continue working with some humans, and delegating security to "someone". That someone could be embedded in the company, or they could be a SaaS+human-support system like zkao.
On the other hand, vibe coders will definitely use claude code security, maybe we should call it "vibe security"? I don't mean it as a diss, I vibe code myself, but it will most likely be as good as vibe coding in the sense that you might have to spend time understanding it, it might make a lot of mistakes, and it will be "good enough" for a lot of usecases.
I think that world is a bit more realistic today, than the AGI "all of our jobs are gone in the next years" doom claim. And as
@zksecurityXYZ
, I don't think we're too scared of that world.
These tools have been, and are making us stronger auditors. We're a small, highly specialized team, that's resilient and hard to replace. On the other hand large consultancies and especially consultancies that focus on low hanging fruits like web security and smart contracts are ngmi.
Developers will not cease to exist. The developers of tomorrow will simply being doing things that developers today can’t possibly even imagine.
Auditors though, they are cooked.
>Auditors though, they are cooked.
I think you're massively underestimating the complexity and depth of a good security audit service.
God bless you, the beautiful thing about computer security is that this attitude has kept us happily in business for many years.
People who don't do intense security work for a living underestimate the complexity of it. This might find some vulnerabilities, but it's not really capable of producing new methods and attacks. What it replaces isn't a high quality human researcher; it replaces current static code review systems.
If AI models never had stack smashing writeups in their corpus, they'd never be able to invent stack smashing.
Asking for a friend who’s working on a startup around this general space: do you think it’s better to go niche, focusing on agents for a specific type of application or a specific language/ecosystem, or is that effectively “killing the startup” by limiting market size too soon?
Another question that came up in conversations with them: there might be value in offering a nonscalable, high-touch service, where you build and maintain customized agents tailored to a client’s specific codebase on a periodic basis.