The AI slop security reporting is basically extinct. It almost does not happen anymore. At all.

I want to emphasize this because when I talk about AI security reports now, half my readers seem to believe those are AI slop. They're not. They are found with AI tools and normally high quality bug reports.

The weakest part is that they tend to overstress the vulnerability angle. Lots of them are well phrased bug reports that are still "just bugs".

@bagder Yeah, seems like around january things flipped around.

I was hoping the slop would continue to be slop, but alas. Wishful thinking on my part (to make it easier to disregard the fad).

@bagder The other problem with AI bug reports is the verbosity, otherwise I basically agree.
@evilpie true they are normally way too talkative

@bagder I get this with fwupd too. Everything that's AI found is reported as a CVSS 10.0 CRITICAL vulnerability, and then you find out it's assuming the attacker has write access on /etc or something dumb like that.

At that point it's just a regular old typo bugfix like all the other thousands of unimportant commits.

@bagder "they tend to overstress the vulnerability angle." which I imagine is simply because that's what the prompt suggested.
@utopiah probably, but also because the AIs can't really tell
@bagder sure, ironically enough there is no "I" in AI.
@utopiah @bagder there's no irony at all, it's at minimum a marketing strategy.
@bagder Well, I guess you could quickly convince them otherwise with your "reports/ai-slop ratio" graph.

@bagder I see
- good ones using AI as part of a rigorous process with replication
- mediocre where someone asked an AI "Find me a CVE", submits the report without review or replication, and yet still expects credit

If "have write access to the filesystem" is a prerequisite to an exploit: it's not an exploit. You already have total ownership of the server

@bagder Do reporters share the tools used, or are there strong tool indicators in the reports?

Curious about which tool(s) are most successful, at least for cURL research.

I imagine in most cases reporters don't mention the tools used (especially if custom), which is unfortunate.

@bagder you're lucky. I got 30+ yesterday. 1 was kind of credible. The others were effectively documented behaviors of projects.
There's still little to no consequences for wasting time - I've been thinking about the "name and shame" approach you have, maybe that helps change the behavior?
@bagder I wonder how much of that is because you eliminated the bounty