What afl-fuzz is bad at | Hacker News Reader

What afl-fuzz is bad at

blog.regehr.org

54 pointsdoty10 comments

I wish there was an afl-fuzz like tool for fuzzing network traffic. By afl-fuzz for network traffic I mean a general purpose network fuzzer; not protocol specific fuzzer like the codenomicon ipsec fuzzer. Does anyone know of anything close? Remotely close?

There seems to be a significant overlap in what you would want from a network protocol fuzzer and a tool to reverse engineer a network protocol. Netzob is the only protocol RE tool that I know of and it seems that development has stalled.

But what exactly are you fuzzing without being protocol specific as you start fuzzing? How do you eliminate false positive? You can be generative with grammar or with some learning capability (as "dumb" as aligning similar traffic as a cluster for example).

I am not sure how you are distinguishing my theoretical protocol agnostic fuzzer from afl. Is afl a jpg/png specific fuzzer? One of the things that is great about afl is that it is not targeted towards a specific file format it can work with a small corpus or thorough dictionary of the target format. eg I can give afl-fuzz a snippet of two of markdown and it will go nuts or if I want to be exhaustive I can feed afl the afl "dictionary" from the commonmark testsuite.

How do you ever eliminate false positives when fuzzing? And what constitutes a false positive?

having only read about the usage and successes of AFL, I'm curious what you would "feed" this network protocol debugger. a grammar file (something like BinPAC++ [0]) or something else entirely?

[0] http://www.icir.org/hilti/binpac/intro.html

A PCAP file of captured traffic?

This was posted a few days back, I don't know if you've seen it? https://lolware.net/2015/04/28/nginx-fuzzing.html (discussion: https://news.ycombinator.com/item?id=9463785)

You can try https://github.com/google/honggfuzz - The basic idea is pretty much the same as with afl-fuzz - maximization of code coverage (more or less).

Some old write-up on the technique can be found here: https://github.com/google/honggfuzz/blob/wiki/AttachingToPid....

You can attach to a process (-p pid) and then feed it with an external, initial input (from pcap, or hand-crafted). Honggfuzz will modify it to maximize code coverage in the network server. I got pretty decent results with e.g. apache (in must be executed with -X, so it doesn't fork/daemonize).

I don't see why afl-fuzz needs to be turned into a general-purpose Swiss Army Knife framework. It excels quite well at mutation-based techniques that train themselves on resulting outputs. For problems that involve specifically triggering code branches, then a tool that employs symbolic execution methods via constraint solving should be used instead, as the author implies.

(afl-fuzz author here)

The basic idea for AFL is that it's supposed to be reliably better than dumb fuzzers without requiring you to think too much about the problem space, fuzzing settings, grammar definitions, harness design, etc.

Anything that makes it retain these properties while improving results is probably worth pursuing. Conversely, anything that sounds cool but ultimately doesn't meet that criteria is probably a better fit for other tools.

The "without requiring you to think too much about the problem space" is pretty huge and it is where solver-based testing currently falls over (in my experience).

Reply on news.ycombinator.com