Hacker News Reader

Comment by viraptor | Hacker News Reader

I can't wait for the time when if your project reads files, fuzzing it is a well-known, recommended practice. (unless you're writing fully managed code)

There were some fuzzing projects before afl, but even with afl's recent popularity we're still in the situation where you just have to grab a random application, point afl at it and get some basic crashes in a few minutes. With clang-analyzer, coverity, afl, and many other projects available for free, there's no reason this should be possible.

Then again, I'm still waiting for the time when people don't code with sql injection issues...

View on HN

Even if you're writing fully managed code you should still fuzz. It can help find logic errors in the code which can cause erroneous program states or crashes.

The fact that the code is managed helps prevent against some nasty exploits, but it doesn't necessarily mean the code always does what the programmer intended.

Agreed! I like to think of the fuzzers as merely an automated tool to drive test cases, maximizing on coverage.

So when asked, "Why fuzz test?" I can answer, "remember when you wrote fifteen test cases and wondered to yourself whether you needed a sixteenth? But then you decided that it would take more work than it's worth to do that. -- Well, now we can tell the computer to just keep trying stuff until it finds unique cases."

And then after that you need to go into formal proofs.

There is also generative testing, which can be a good middle-ground between hand-crafted tests and fuzzing. You can think of it as a fuzzer with more guidance regarding input generation, and can uncover edge-cases that a fuzzer would take forever to discover. I tend to think of those 3 approaches as complementary.

Example in Clojure: https://github.com/clojure/test.check

Sure, I mean fuzzing should be critical for applications which can sigsegv, but it doesn't mean others shouldn't also give it a go. It just requires some integration support (you need to let the fuzzer know that an exception got all the way to the top for example), so it's up to the project if they want to go that way.

There were and are a lot of fuzzing tools before AFL, it's been the normal go-to approach for around 10 years for finding security bugs.

Because most software projects are so lazy about doing fuzzing themselves, making and promoting easier to use tools is a good way to nudge people into picking the low hanging fruit (bugs?).

Fuzzing works for managed code, but since you are looking for logic bugs instead of memory corruption crashes, more effort is needed to verify the behaviour of your app in the face of fuzzed data. Things like TOCTOU bugs, data sanitation bugs like path traversal or SQL injection vulns, etc. Of course this finds non-crashy security bugs in C code too, like in the Heartbleed case...

To expand on the earlier fuzzers, Section 2.2 and Appendix 1B of http://www.dtic.mil/dtic/tr/fulltext/u2/a558209.pdf seem to have decent information of the noughties tools.

I'd extend that to say not only "reads files" but "consumes any input".

One of the things that made me a little twitchy is that FFmpeg replaced the very thoroughly fuzzed official Opus decoder with their own implementation that wasn't.

It's actually good to have multiple independent implementations of a decoder to find inconsistencies and bugs. In fact Mozilla sponsored the work, and Google also did indirectly through GSoC.

Why are you not waiting for secure software instead of being specific like that (SQL injections)? Do you think that it will never happen?

I can't wait for the time when it isn't considered ok to use languages that are so error-prone. I wouldn't write anything in C.

Reply on news.ycombinator.com