Fuzzing vs. Property Testing

Of interest is Crowbar[0,1], a testing tool for OCaml that combines Quickcheck like property testing with AFL fuzz.

[0] https://github.com/stedolan/crowbar

[1] https://ocaml.org/meetings/ocaml/2017/extended-abstract__201...

One of the things that I like about go is that the standard-library really encourages the use of writing test-cases. Most of the bigger projects have good coverage, and I try hard to get my own.

But nothing beats the sheer tenacity of running your code through a fuzz-tester. I've written a simple virtual machine, which interprets bytecode, and a BASIC interpreter amongst other things recently. Both of these projects benefited hugely from fuzz-testing, despite having high coverage via manually-written test-cases.

Fuzz testing is cheap and largely automatic, so if it takes a few hours or a few days to find an interesting result that's not a problem. I remember the first time I tested my interpreter when I had ~90% coverage of the code with my test-cases and it crashed via fuzzing within seconds! Magic!

Related, I've been having a lot of success with Hypothesis [0], the Python property-based testing library.

[0] https://hypothesis.readthedocs.io/en/latest/

A good property testing framework has two properties, 1) inputs increase in complexity over generations, and 2) properties can shrink, enabling you to find "minimum error conditions", so property tests can help you identify the source of the error too.

Is there something like a good proptest in C++?