Hacker News Reader

Comment by yaantc | Hacker News Reader

You have two kinds of fuzzers:

1) out of process, like AFL: The application is launched for each test, and there's no problem handling a crash, whatever the cause (asserts are ok). But for each test there's the application start-up (process creation, etc.) overhead.

2) in process, like libFuzzer: The application is launched once only. Then inside the application context the library iterates over the tests. So no application start-up overhead (nice), but a brutal exit is a problem. That's where asserts becomes a problem.

I guess that's what's at play here.

View on HN

Why not change asserts (they are macros after all) to tell the fuzzing library that it found an error? Finding bugs is the whole point of the exercise, isn't it?

From a quick look at the doc (http://llvm.org/docs/LibFuzzer.html) this doesn't seem to be supported.

Aborting from an assert deep in the code has some challenges: unwinding the stack is ok, but how one would avoid memory leaks for heap data? It would require trapping all mallocs to track all allocated data and free it. Not impossible but it adds very significant complexity, and that doesn't seem the philosophy of libFuzzer. It seems to me that the goal of libFuzzer is to be very easy to use (just define one function for test harness, compile with CLANG and fuzzying flag, done) and efficient on an already reasonably well behaved library.

It makes sense to me: just start with AFL, which has no problems with crashes. When you reach AFL limits and need more efficiency, only then move to libFuzzer (both can shared state / test framework). Then the assumption that the app. is reasonably well behaved makes sense.

Caveat: I haven't used libFuzzer yet. For the reason above, I'm just using AFL for now. Maybe one day but I'm not there yet ;)

> But for each test there's the application start-up (process creation, etc.) overhead.

That's not completely true, for AFL you have __AFL_INIT(), which defers the initialization of AFL until after an expensive initialization of the program to be fuzzed, which greatly improves performance.

Reply on news.ycombinator.com