A Fuzzer for the Toy Optimizer
https://bernsteinbear.com/blog/toy-fuzzer/
#HackerNews #Fuzzer #ToyOptimizer #SoftwareTesting #Programming #Security
A Fuzzer for the Toy Optimizer
https://bernsteinbear.com/blog/toy-fuzzer/
#HackerNews #Fuzzer #ToyOptimizer #SoftwareTesting #Programming #Security
Published my first PyPI package today, called lafleur.
#lafleur is a specialized CPython JIT fuzzer that uses a coverage-guided, evolutionary approach. It executes test cases, observes their effect on the JIT's state by analyzing verbose trace logs, and uses that feedback to guide its mutations, becoming smarter at finding interesting code paths over time.
Let me know if you use it or have any questions.
https://pypi.org/project/lafleur/
https://github.com/devdanzin/lafleur
There's a researcher, Jiang Yuancheng, who's doing a great work finding CPython crashes and memory leaks: https://github.com/python/cpython/issues?q=is%3Aissue%20author%3AYuanchengJiang
They've come up with a very clever idea for a new way of fuzzing, made a fine tool out of it, and are reaping great results.
Fuzzing can be a diminishing returns endeavor: you only have so many bugs to find. Their approach has shown itself to cover different areas and kinds of issues well, as shown by their track record.
"Tạo công cụ kiểm thử AI Red Teaming cục bộ với Fuzzer và 280 Payloads để tránh đăng ký hàng tháng. Công cụ này bao gồm Mutation Fuzzer, Payloads thực tế và AI Judge. #AITools #RedTeaming #Fuzzer #AI #AnToànBảoMật #CôngCụKiểmThử"
https://www.reddit.com/r/LocalLLaMA/comments/1p6c21o/i_built_a_local_ai_red_teaming_tool_fuzzer_280/
It turns out that, by running it on an interpreter with ASan enabled, I was the culprit of my fuzzer lafleur using way too much memory.
That even led to me buying some DDR5 to be able to fuzz a bit more comfortably. Running without ASan reduces memory usage to 1/15. So I guess now I'll have some spare RAM, and less money, going forward :)
I'll enhance the JIT fuzzer to run on a different interpreter than the fuzzing scripts, which benefit from ASan.
So someone found a segfault in NumPy and reported it as a security bug: https://huntr.com/bounties/49928a2c-c6bb-4c1c-80ec-5d7bf708bf28. After some back and forth, the NumPy developers agreed it was a security bug (with a low score, but still).
However, since fusil had already found that crash and I had reported it 2 months earlier (https://github.com/numpy/numpy/issues/28829), the report was deemed a duplicate and no CVE was assigned.
We didn't find a CVE, but avoided one :)
Link to fix: https://github.com/numpy/numpy/pull/30071
Fuzzing pyhacl (https://codeberg.org/drlazor8/pyhacl), a package of Cython bindings for HACL* (the High Assurance Cryptographic Library), with fusil we only found one crash.
It turned out to actually be a silly bug in #Cython:
Issue: https://github.com/cython/cython/issues/7263
Fix: https://github.com/cython/cython/pull/7264
Goes to show how fuzzing a C-extension can uncover crashes in many different layers.
Thanks @drlazor8 for taking up the call for C-extensions maintainers to fuzz their code.
After a pause, we're back to running fusil. This time, to fuzz cereggii, a package of very interesting thread synchronization utilities for #Python, made of C-extensions.
We tailored #fusil to target these #cereggii utilities, finding more issues.
Daniele Parmeggiani (dpdani), the maintainer, has been helping in the effort and being very supportive. That's the best welcome fusil has received in any project :)
Here are the issues we found: https://github.com/dpdani/cereggii/issues?q=label%3A%22fusil-fuzzer%22