FuzzBench: Fuzzer Benchmarking as a Service

They measure unique edge throughput AFAICT. I wonder if there's a fair way to evaluating fuzzers by throughput of unique failing test cases instead.

I hadn't heard of most of these fuzzers. MOpt [1, I think] seems like it has the performance crown (based on [2]).

> Inspired by the well-known optimization algorithm Particle Swarm Optimization (PSO), MOPT dynamically evaluates the efficiency of candidate mutation operators, and adjusts their selection probability towards the optimum distribution.

This is a great idea. Intuitively, each program could respond differently to different test case mutation.

[1] https://www.usenix.org/system/files/sec19-lyu.pdf

[2] https://www.fuzzbench.com/reports/sample/bloaty_fuzz_target_...