Announcing OSS-Fuzz: Continuous Fuzzing for Open Source Software

I wrote a wrapper around CPython for fuzz testing with libFuzzer [1]. It's dog slow because it's designed for anyone to be able to write relatively simple test cases in Python (so every iteration of the fuzzer starts the interpreter, executes the test, tears it down...). But a neat concept IMHO.

[1] https://bitbucket.org/ebadf/fuzzpy

We developed a fuzzer [0] using property-based testing in Haskell using QuickCheck (as expected!)

[0] http://QuickFuzz.org/

I developed a fuzzer that generates random values based on ABNF rules, such as often appear in IETF RFCs. So it can be used for testing RFC implementations. It's written in Java but can be called from other tools.

https://github.com/nradov/abnffuzzer

There is a good wrapper for afl, https://pypi.python.org/pypi/python-afl that allows you to relatively easily use afl with your python code. Just be sure to take advantage of looping or you'll be limited by the python runtime start-up overhead.

Seconded. I've used it for testing various functions in recommendation systems, and it's great at finding boundary conditions you didn't think of, or issues with floating point math. And, as it was the first time I used any testing tools like this, it was a fun emotional experience to have an antagonistic test suite.

Since people are listing fuzzers, there are also the commercial alternatives: Peach Fuzzer (www.peachfuzzer.com/) and Synopsys Defensics (https://goo.gl/wroKI8‎). While my experience is not that recent, these two seem to dominate the network protocol fuzzing domain, whereas some of the open alternatives listed here appear to have become superior in general purpose 'parser' fuzzing. That said, played with honggfuzz recently and it really shows promise, especially if you have access to the source code (https://github.com/google/honggfuzz).

Adding a new project doesn't seem too tough. Guide: https://github.com/google/oss-fuzz/blob/master/docs/new_proj...

> what alternatives are there to making the libraries it checks more secure besides rewriting them in another language?

I'm not sure what you mean by rewriting a library.

For example, if a library is written in Ruby, fuzzing might not be suitable. you shouldn't rewrite it in C, so that you can fuzz it. (I thinkI misunderstand your point here. Please correct me.)

If we want to discover vulnerabilities in Ruby, we should fuzz the Ruby VM directly. We shouldn't fuzz a Ruby library.

Why wouldn't they be happy? Google's a diverse company, it's not like every department/team has an unwritten goal to push Chrome at the expense of everything else.

Kostya would be quite happy to have firefox using it :)

Google actually did and does a whole lot to support firefox security. There's a lot of collaboration in the security space, e.g. Firefox and Chrome security teams heavily collaborate on fixing the TLS/CA ecosystem.

Also the interest is very often shared, because a whole pile of libraries are used by both browsers: freetype, libjpeg-turbo, libpng, giflib, expat, webrtc, opus, ...

I think that represents a fundamental misunderstanding of Google's incentives. (disclaimer: I'm typing this from Google, but it's purely my own opinion.) (And wishing my experiment would finish running faster.)

Google's biggest revenue stream is ads. It has many others, but that's the big one. Ads are seen by people who use Google search, and who browse many ad-supported websites. Google also has a lot of users of its services, e.g., gmail, who have accounts. Some also entrust valuable data to Google.

Google therefore has a great deal of incentive to make sure that: (a) Nobody messes with the ability of general users to browse the Internet safely (preserving ad revenue); and (b) It's very hard to compromise users computers and gain access to their Google/gmail/whatever accounts (reducing support costs and keeping users happy), or destroy their data, or exfiltrate it. Even if the compromise was the user's computer, it's still a very bad experience.

Project zero is a pretty good example of this incentive structure in action. (such as fuzzing for windows font bugs: https://googleprojectzero.blogspot.com/2016/07/a-year-of-win... ).

With fuzzing itself, as DannyBee alluded to below, one of the leads of a lot of this infrastructure, Kostya Serebryany, is also personally passionate about seeing it be taken up to make the software of the world better. I've seen this in action - he convinced me to use libFuzzer to improve TensorFlow's robustness, and mentioned that he was doing so in part so I'd take that experience back to Carnegie Mellon and spread the word. :) (and it worked - https://github.com/tensorflow/tensorflow/commit/7231d01fcb2c... for example).