Linux Filesystem Fuzzing with American Fuzzy Lop [pdf]

Very much so. Even really common code has issues found via fuzzing on a regular basis. In the past six months I've reported NULL pointer deferences printing fingerprints of SSH keys, segfaults in the GNU awk parser, and most recently segfaults when parsing HTML files with w3m.

Most of the time setting up fuzzing is pretty simple, but even updating programs to read from STDIN rather than sockets is usually possible if you're careful and patient.

The hardest part is waiting for the damn things to run. Right now I've had a fuzzing session going against the text-based browser lynx for 4 days, and still counting. (Specifically looking to see if it can be crashed when converting HTML to text, as is often done in mutt, etc, via 'lynx -dump').

ZFS is not included in the Linux kernel. Much of this work was about building the infrastructure to be able to perform this kind of test.

Trying to do it for file systems that aren't included in Linux is a lot of effort for much less gain.

I was able to set it up in a few hours for a moderately sized library. Along with valgrind, I was able to find and fix all of the bugs it uncovered after over a CPU month of testing.

If you assume that, you don't need checksums in the first place.

Of course that means your mutations have to be "fair" in the sense that your new values should still be valid. Because as you stated, just stomping on blocks and fixing up their checksums seems like a silly test.

For example (on a completely hypothetical filesystem/disk), lets say you want to open a file. You read the entry in the filesystem metadata that tells you on what sector that file begins and it gives you a value of 5005. Even if the checksum is correct, the filesystem code still needs to check that a) 5005 is actually within the range of valid sectors, b) that it actually makes sense as a sector value (maybe files must always start on a multiple of 4), c) that the data at that sector actually looks like a file, d) etc.

If you don't update the checksum as well when generating the test data then you limit the amount of depth in the testing since most errors would get caught immediately at the checksum validation stage.

That would be absolutely unreasonable. Filesystems can be malicious, for example Stuxnet transmitted via USB flash drive.

A reverse time-sort would be

    ext4 (2h)
    XFS (1h45)
    GFS2 (8m)
    NTFS (4m)
    NILFS2 (1m)
    HFS (30s)
    HFS+ & ReiserFS (25s)
    OCFS2 (15s)
    F2FS (10s)
    BTRFS (5s)

Even in the Btrfs case, it may just be a function of the code having lots of awareness of where bodies are buried, so it hits BUG rather than continuing in a bad way and blowing up later. But as yet not enough testing to figure out all the path ways for hitting those bugs. It may explain the edge cases that still pop up with Btrfs, where I go "umm yeah I use this every day and never experience anything like that" but another user suddenly hits something that's a "wow" moment.

You can find the source code here: https://github.com/thoughtpolice/afl/commit/e54c0237e934d734...

It should not be difficult to update this to work on more GCC versions (I only tested on GCC 4.8.x), but that will take some #ifdef'ery. Porting to newer AFLs should be relatively trivial.

EDIT: I initially wrote this for no particular reason, mind you, other than to play around with writing GCC plugins, and the result wasn't so bad, modulo non-existant documentation. I also thought it would be nice to have an identical equivalent to 'afl-clang-fast' for GCC ('afl-gcc-fast'), in the hopes that perhaps one day the hacky, sed-inspired backends could be removed from afl. I initially wanted to use this on a POWER machine as proof of a portable GCC plugin for afl, although I lost interest in porting to a newer GCC, before losing access to the machine. Watching afl fly on 176 cores was fun, though.