Show HN: Differential Fuzzing of Cryptographic Libraries | Hacker News Reader

Show HN: Differential Fuzzing of Cryptographic Libraries

guidovranken.com

84 pointsguidovranken12 comments

I've posted this once before but it failed to get traction. I'm reposting this with the permission of HN admins.

In a nutshell: This open-source project provides a framework for fuzzing cryptographic libraries. It can find both memory bugs and implementation errors. So far it has found about 35 bugs of varying severity in the stable releases of major cryptographic libraries. The fuzzers run continually on Google's OSS-Fuzz platform which helps detect bugs before they end up in stable releases.

The article describes the technical details of the software. Here you can find a summary of the bugs it has found: https://github.com/guidovranken/cryptofuzz#hall-of-fame

Thanks for posting this. It looks like some really great tooling. I tried fuzzing the Argon2 library a while ago and very quickly hit some of the issues you appear to have addressed.

What tools/library you used to fuzz Argon2?

In the Bitcoin project we found and reported several bugs in OpenSSL based on similar harsnesses that compared our code with similar functions in OpenSSL (where they exist).

It's a pretty powerful technique at least in problem domains where fully specified behavior is a realistic expectation.

Mutation testing-- where you break the code and confirm that your tests fail-- is also pretty powerful in those same domains.

(in particular: mutate the code, if the tests pass, test random cases differential with the original, if any discrepancies are found then you just found a missing test case.)

Differential fuzzing (and the closely related metamorphic testing) are amazing ideas, I've used them in three different ways:

1) Compare two programs

2) Generate two inputs for one program which should produce the same output

3) generate two inputs for one program where there is a simple relationship between the outputs.

In all cases, it has been a great way of testing, and has shuck out some incredibly subtle bugs I don't think we would have found any other way.

I use the high level concept pretty regularly in my day-to-day as a security consultant specializing in cryptography, and this project is a fantastic way to democratize the use of differential fuzzing. The only negative thought I have about this is that I didn't think of it first!

dumb question: but why is it so hard to create very-secure crypto libraries? i recall NASA's code used during the Apollo missions had like an ungodly low error rate, so surely it's feasible in practice? admittedly, comparing anything to NASA standards is a very high bar to set, but considering the stakes involved with cryptography, you'd imagine a similar scrutiny?

Apollo computers had input that was trusted, and predictable (within ranges). It only had one job to do. Albeit a big job.

Crypto libs (or any libs) typically can be used in a vast number of ways with completely untrustworthy input.

I would add that in the beginning, a number of crypto libraries (I'm looking at you in particular, openSSL) made a design decision to support a huge variety of algorithms, options, modes of operation etc. This makes it very difficult to test the full functional surface of the software and means you just have to write a very large amount of code.

Some of the modern libraries take a different approach where they deliberately limit the functionality to a few select things that can be implemented easily (thinking nacl, libsodium, boringSSL etc here) and which guide the user towards sensible defaults that work rather than having huge numbers of levers to tweak, only some of which arrive at a secure and supported solution.

The government has very deep pockets. And working for the biggest project of the century probably attracted decent talent, whereas there's no glory in writing supporting libs - you either did it right and nobody notices, or you messed up and get blame.

Please don't make screenshots of text.

Reply on news.ycombinator.com