Hacker News Reader

Comment by yjftsjthsd-h | Hacker News Reader

> Close to half a billion fuzz runs found 5 runtime panics, all of which were detected by Rust before they could compromise security. If I'd written this code in C, several of those errors would have been exploitable.

Wouldn't the apples-apples comparison be fuzzing a C program? If AFL would catch the same bugs, Rust isn't better than C in this case.

View on HN

> If AFL would catch the same bugs, Rust isn't better than C in this case.

You can find a list of my vobsub bugs in the Rust Fuzz Trophy Case: https://github.com/rust-fuzz/trophy-case They are:

1. The shift overflow: This was harmless in both C and Rust, I believe.

2. The arithmetic overflow: This was a runtime panic in Rust, resulting in a clean crash. In C, this might have been exploitable with a complex enough attack.

3. The three "invalid slice" errors. All of these were clean runtime panics in Rust, and I'm pretty sure that at least two of them would have been exploitable in C with enough work.

So Rust has a couple of advantages here. Out of 5 errors found by the fuzzer, the worst thing that could happen in Rust would have been crashing the program in a controlled fashion. In C, we would have been looking at memory corruption and quite possibly escalation.

Secondly, Rust's runtime checks actually make fuzzers work better. Even simple checks like "is the end index of this slice great >= to the beginning index?" tend to catch problems almost as soon as they occur.

I think you aren't understanding what parent comment is trying to say. No one is saying rust isn't better than C in some respects, instead, you were comparing Rust + Fuzzing with C and he implied that you should at least use C + AFL if you want to compare properly otherwise it's just BS.

It's a pointless comparison. Fuzzers don't catch everything. What's important is the type of issue that the fuzzers do reveal, because that gives a hint as to how severe the uncaught issues may be.

You would get 2. and 3. as runtime panic in C if you used sanitizers.

You would also get additional attack surface if you used sanitizers, because they were designed as debugging tools, not for production deployment.

http://seclists.org/oss-sec/2016/q1/363

The point here is that a runtime panic is not an exploitable event - at the very worst, it's a DoS as it usually results in the program exiting.

AFL isn't even nearly guaranteed to catch everything, but what it's shown here is that the program contained bugs which would've resulted in an exploitable event in C. The program probably contains more bugs, but following this pattern, it's more likely that they result in panics in most of the worst cases, not exploitable events.

With Rust, a panic occurs right away when the fuzzer develops input that performs e.g. out of bounds access. With C, it's possible to go wrong a little but not enough to segfault or to go wrong enough that by accident the access looks valid (for an unintended valid memory address) to a sanitizer.

In that sense, fuzzing Rust is more certain to make the bad things actually be caught than with C where there's a chance that the fuzzer developed input that caused UB but the UB didn't happen to be caught by the mechanisms that are supposed to discover bad events during fuzzing.

The difference is not in what was caught by AFL, but what wasn't (and also, what would have happened in C had AFL not been run).

One implication is that C would have had more errors, but that's a hard judgement to make. It's probably true.

The thing that really matters is that these errors which the fuzzer found aren't exploitable -- memory errors in rust are guaranteed to panic (unsafe aside) instead of susceptible to overflows, etc. Since fuzzing is very much more of an art than a science, this guarantee is important.

Reply on news.ycombinator.com