Mastodawn

Dipped my toes in #AFLplusplus over the weekend was a kinda nice distraction / pastime.

Got #moarvm (language runtime for #rakulang) its first fuzzing target that exercises the streaming decoder system which is what you would use when reading strings (rather than bytes) with some encoding from a streaming source like the output of a process you're running.

Was a big help to actually put rules in the Makefile to generate the different variants (one with cmplog, one with asan, one with lcov, one with just afl instrumentation) of all the .o files, but I'm not so happy with the rules I came up with.

And to add to the pain, of course there's always nmake which I don't feel like figuring out what exactly I have to do to replace the usage of addsuffix I have ...

https://github.com/MoarVM/MoarVM/pull/1996

If you want to chime in with ideas for what would be good to do with fuzzing in this context, send a comment or toot!

Giving MoarVM its own instrumentation for moarvm bytecode for path coverage would be cool for the future!

Show thread

Timo the timo

Seeing the explanation of how the path coverage instrumentation that comes with #AFLplusplus works made me realize that it would actually be not just possible but even probably feasible to build something that's just "fully compatible" with what AFL expects to receive.

It's really just a large array of byte values, the address and size are supplied by AFL, where you increase a value at a position when a given edge has been traversed.

The number any given edge gets is just every relevant BB's number xor'd with the previously seen BB's number shifted by 1.

What number do you give a BB? Uniform random numbers seem to be good enough for a start!

An open question I have is that we do compile and then run some code at compile time when compiling #rakulang code, so ideally the "same" input code (for a suitable definition of same) would get the same numbers for all BBs so that only consequential changes in behavior are considered interesting by the fuzzer ...

Show thread

Timo the timo 1d ago

On the other hand, unless the freshly compiled code doesn't interact with any code that was comes from a file on disk (and is therefore easy to construct stable identities for BBs for) it will still leave different traces in the code it itself calls.

If it actually behaves the same as in a different run, then we do want that to count as the same ...

Even if behavior inside the given dynamically compiled code differs for different runs, if the things it calls are the same, how much value is there in feedback for its coverage?

In general, it's fine (and even recommended) to not instrument parts of the program you're not interested in getting better coverage for, and so far for the fuzzing target that exercises moarvm's streaming encoder I didn't take the time to build an allowlist or denylist.

Much to think about, and it would be fantastic to hear from other people who have built their own instrumentation for fuzzing purposes, or users of fuzzing with high-level languages with complex-ish language runtimes!