If your Open Source project sees a steep increase in number of high quality security reports (mostly done with AI) right now (#curl, Linux kernel, glibc confirmed) please tell me the name of this project.
(I'd like to make a little list for my coming talk on this.)
Apache httpd, curl, Django, Firefox, glibc, GnuTLS, Haproxy, libssh, Linux kernel, python, Temporal, Wireshark, wolfSSL
More?
Updated:
Apache httpd, curl, Django, Elasticsearch Python client, Firefox, git, glibc, GnuTLS, Haproxy, Immich, libssh, Linux kernel, OpenLDAP, PowerDNS, python, Sequoia PGP, Temporal, urllib3, Wireshark, wolfSSL
We can say with certainty that this is widespread.
Summary A time-of-check-to-time-of-use (TOCTOU) race condition in TIFFClientOpenExt() (libtiff/tif_open.c) causes a heap buffer overflow when the name argument points to a shared mutable buffer that is concurrently...
@bagder random anecdote tangentially related but I needed to debug a binary on Windows with no source. Claude used nothing but deno as a disassembler and found the exact issue (an async flag where it shouldn’t be and misuse of win32) which saved me hours waiting for the client to “maybe” give me the source.
Claude can be used very well for security work in the right hands.
The next months I will call the-open source--security-apocalypse-dark-times (of death).
Because I wanted a cheerful name that makes it not seem as bad as it is. /s
Should all responsible software be running security agents against their own software now as we do fuzzers/static analysis/tests/etc?
Or instead of being proactive do nothing? And hope it’s not just the nice people reporting the bugs that are finding the issues?
@bagder OpenLDAP is seeing more AI-assisted bug reports that claim to be security issues, but aren't.
E.g., calling a crash in a commandline tool a DoS (no, it's not a service).
@bagder the other one we see is calling assert failures crashes. It's not a SEGV, there's no possibility of data exfiltration or RCE. There's no security exposure, it's just a bug. One that was anticipated hypothetically by the original developer, but whose final disposition wasn't decided upon way back when.
E.g. /* can this even happen? */
They toss in an assert, and it lives quietly in the code for decades before someone definitively shows yes, it can happen...
@bagder @fightbackman So Carlini, the Anthropic guy, is not just a salesman? Serious question.
For me it's still open how much of a flood of reports should be expected.
Just so I understand this correctly...
We don't want machine generated vulerability reports...
...so we can leave our #foss projects vulnerable to hackers who are not constrained by ideology in their sploits using #Ai ?
Yeah, that tracks with the current majority of #infosec "professionals" letting the Rome burn while they roast the marshmallows, feeling super pure and superior.
Your talk is going to be super fun.
Send me a link please!
@n_dimension @bagder The projects typically want security/bug reports, not computer generated words that *look* like security/bug reports.
Same reason you don’t want parrot operating your air traffic control tower radio. Do you want an air traffic controller or a parrot that sounds like an air traffic controller? Do you trust the parrot to safely direct planes according to aviation regulations?
Even a broken clock is right twice day.
Lucky then the project maintainers don't have to be bothered by minutea of securing their projects with automation...
...because #blackhats certainty don't have the same reservations.
#Ai is a new attack surface and acting irrational and emotional towards it is incomprehensible
@RoganDawes @ClickyMcTicker @bagder
Time flows independent of the perception of the observer, therefore the timepiece is correct twice as time in the 24 period is linear and constant.
Huh, what do you know, it is 99.8%
@n_dimension @bagder let the attackers spend their time sifting through AI slop trying to find legitimate vulnerabilities. The defenders have a difficult enough time dealing with real, validated vulnerabilities.
If you want to spend your time proving us wrong, feel free to run your favorite LLM against a FOSS tool, then manually validate that what it spits out is legitimate by writing a proof-of-concept that is exploitable in the real world. If you do that, I’m sure the FOSS project of your choosing wil fix it.
The issue is that the flood of AI generated false positives are more than the all volunteer team of folks supporting a FOSS project can handle. aI is great at writing convincing slop. It has not demonstrated its ability to consistently find legitimate vulnerabilities.
If you disagree, prove us wrong… but you do the validation yourself. Don’t just spit out AI slop and make someone else do that work.
I'll just leave this here
https://news.ycombinator.com/item?id=47633855
...there is no "sifting" here
@n_dimension @bagder Tell that to the FOSS maintainers who receive hundreds of fully AI-generated "vulnerability" reports that all turn out to be false positives.
If you want to use AI to find a bug, go for it. Validate the bug. Write a proof-of-concept (or have AI do it if you're not capable) and test it yourself. If your proof-of-concept achieves the desired results, then submit the bug and the POC.
There are people just haphazardly feeding FOSS baselines into local AI and asking for bugs, then submitting whatever their LLM tells them without validating that it's correct. This effectively floods the maintainers with false positives and makes it very difficult for legitimate bug reports to get through.
Also, just because Claude found a bug doesn't mean it didn't also report 100 false positives before it found a real one. Given the effort it takes to triage a bug report, allowing any random yahoo with a keyboard to blindly submit AI-generated slop equates to enabling a DDoS on your bug triage staff. It's not sustainable.
@n_dimension @bagder Just so I understand this correctly...
you don't
Then assplain it to me kid
@n_dimension @bagder first you change your tone.
Then please explain which part of "If your Open Source project sees a steep increase in number of high quality security reports (mostly done with AI) right now (#curl, Linux kernel, glibc confirmed) please tell me the name of this project. " you don't understand resp. where do you see something indicating not wanting machine generated reports.
Are these legit sploits or noise?
@n_dimension @bagder I asked "where do you see something indicating not wanting machine generated reports"
Can you please answer that question.
daniel:// stenberg://
🪨
hsbt
Anders Thoresson
Even Rouault
kby
Brændꜵn Mollɔy 
René Dudfield
screwlisp
Clemens
Frederik Braun �
Howard Chu @ Symas
Cassandrich
Nantucket Lit
Jakob Borg
Stefan Eissing
pettter
M Schommer
fightbackman
goedelchen
Björn
Wulfy—Speaker to the machines
Mathaetaes
Space Invader
Julien W.