Hailey1d ago

so it cost anthropic $20k to find this openbsd crash bug which amounts to putting a negative integer in a tcp field where a negative integer was not expected by the c code which does some cavalier int cast bullshit, ie. a vuln which is totally fuzzable, and quite certainly would have been found by the fuzzers of the 2010s had anyone cared to burn that much compute on fuzzing openbsd.

The difference today is not that anybody suddenly cares about investing that much in openbsd (is the build server still a donated machine running in Theo's basement?), but that openbsd's reputation for security makes it really good marketing if you can find a bug, any bug, it doesn't matter; and that marketing value is what makes it worth spending $20k on fuzzing.

Show threadDavid Chisnall (*Now with 50% more sarcasm!*)18h ago

@hailey

It would be interesting to see if Coverity found it (and even more interesting to see if Coverity reports were part of the training set).

FreeBSD was given a free Coverity subscription but it generated enormous numbers of reports. I went through the ones for bits of code I’d touched and they were almost all issues causes by not understanding code across complex control flow (particularly things invoked via function pointer). I think one was a real bug, out of dozens I looked at.

Paying someone $20k to go through and triage as many Coverity reports as they could in however long $20K buys of a competent person’s time would almost certainly have found and fixed more bugs.

EDIT: Coverity does scan OpenBSD but the results are not visible to the public. Any OpenBSD people able to check whether this bug was in the last scan report? Anyone else know whether Coverity scans are in Anthropic’s training set (maybe they just bought a Coverity license and did their own scan of a load of projects for training data?).

Show threadJohn-Mark Gurney8h ago

@david_chisnall

@phessler could you check? Or maybe someone else?

@hailey

Show threadDavid Chisnall (*Now with 50% more sarcasm!*)

@encthenet @phessler @hailey

Colin checked for FreeBSD and apparently Coverity didn't find the reported issues.

5:14pmWeb