Hacker News Reader

Comment by tptacek | Hacker News Reader

This seems like a long axiomatic rationalization for using memory-unsafe languages. "[W]ith the right tooling, practices, and discipline, one can reduce the risk of memory safety bugs to an acceptable level for their situation". Well, empirically, no, that's not the case: the largest, most sophisticated teams in the world have tried, and it hasn't worked out.

View on HN

>> Well, empirically, no, that's not the case

Rust itself has had more CVEs in the past 18 months than Postfix (written in C) has had in the past 18 years [1].

So it must be more nuanced than you're claiming.

EDIT: Postfix is somewhere in the region of 1,000x to 10,000x more popular than rust - it's hard to compare a compiler to a mail transfer agent but the MTA is in the more risky use case and yet fares better despite having more sysadmins deploying it than there are rust developers.

[1] https://www.cvedetails.com/vulnerability-list/vendor_id-1902...

https://www.cvedetails.com/vulnerability-list/vendor_id-8450...

There are standard criteria for what can become a CVE, and I guess some Rust users are very zealous with bug reporting. At first glance, it looks like a lot of these issues are minor, but not normal, bugs to see in a programming language. I guess abnormality rises to that level.

A lot of Rust CVEs are filed for things that C/C++ software would just consider normal bugs, or not bugs at all.

For example: https://blog.rust-lang.org/2022/01/20/cve-2022-21658.html

The C++ stdlib is affected by the same issue, but they don't consider it a security bug, because the API doesn't claim to be safe of race conditions. Rust doesn't explicitly claim that either, but the Rust community has a higher bar for correctness around these things, so it gets a CVE.

Prior discussion here:

https://news.ycombinator.com/item?id=30461036

https://www.reddit.com/r/cpp/comments/s8ok0h/possible_toctou...

You're comparing a programming language's standard library with an application. In the former case, most of the CVEs are pointing out bugs in 'unsafe' implementations of std APIs that result in UB where they should not. In the latter case, the CVEs are mostly pointing out actual exploits that permit privilege escalation. If you applied the standard by which CVEs are created for Rust to C---an apples to apples comparison---then most APIs in libc would have CVEs filed for them and they would be marked as 'wontfix'.

It's not at all a direct refutation of the GP's claim.

I'm sorry, but you don't understand how Rust's safety guarantees work. There would be multiple CVEs filed against almost every single library in libc and C++'s standard library if we used the same standards as Rust for those two. We don't, because it would obviously be pointless to do so, because C and C++ libraries generally don't have explicit safety guarantees, and those that do usually put an extremely high correctness burden on the caller. AFAIK, virtually all of Rust's CVEs found to date would go away (assuming the rest of the code remained the same) if Rust libraries just declared all the APIs in question "unsafe," which is basically what C and C++ do. The Rust standard library, your "random" example here, would have 0 CVEs by this metric, and so would pretty much every Rust library (the vast majority of Rust CVEs are filed against libraries).

In other words--a library can't be blamed for being insecure if it never makes any security promises in the first place. Rust libraries make stronger and more explicit guarantees, and the code is encapsulated in a much smaller region of the program, so it is easier to prove that they are in violation of their contract; but it emphatically does not mean that it results in more end to end vulnerabilities. Claiming that Rust simply isn't used in enough, or critical enough, places yet to have real exploits like C programs do is pretty naive, as it's started to be used in places like Android's bluetooth stack.

In fact, in practice there's even more of a gap than this--most C and C++ programs don't report CVEs even for reliably reproducible instance of UB unless you can demonstrate a full end to end attack chain, and many won't even fix or prioritize them if they consider it "benign" UB (what this means is anyone's guess!). Even projects like Chrome that actually care about security often won't upgrade UB bugs to CVEs if they believe (erroneously or not) that other hardening mechanism in the program will prevent it from being used as part of an RCE chain. Applying the same standards people typically do to C and C++ projects (an end to end exploit chain), I'm not sure whether there would be a single known Rust CVE in any widely used program.

Basically: the standard of evidence for a CVE in your average C or C++ program is astronomically high, while in Rust it is extremely low. Any attempt to decide which language is practically safer by counting CVEs (which is in any case bad methodology) needs to have awareness of this fact if it's trying to actually measure anything real.

I mean, you're not wrong, but cherry-picking two examples that aren't really comparable isn't a good argument

Agreed. It's hard to find examples of successful large Rust projects to compare against. The compiler and maybe deno are about it i think?

This is not what you'd expect for an alleged developer rocket ship after 10+ years. It doesn't compare favourably with C++ at 10 years old, or Java, or Go, or...

Largest Rust systems aren't public, like both Dropbox server and client.

Millions of lines of Rust are shipping in Firefox, so we could count CVEs in Firefox attributable to Rust. I agree this is annoying, but someone should do this.

I point out Rust in Firefox is an order of magnitude (10x) larger than rustc or deno.

It's not even millions yet, and Firefox along with Mozilla is already dying. When will the cargo cult end.

Please... I ran line counter on mozilla/gecko-dev just before writing that. There are three millions. Go clone it and count yourself.

It sounds like you just have a bone to pick here, but I would like to point out that C++ came out of Bell Labs, Java had massive corporate backing including a huge marketing campaign and Go has Google's backing. Each of your examples had huge advantages unrelated to any technical benefit.

A closer comparison might be Python, which took a long time to reach its potential, and I see Rust on the same trajectory, with high-profile teams now adopting it, like AWS.

To be clear, if you're using number of CVEs to determine whether some software is more secure than other software then you are very directly incentivizing people to not file CVEs. Your actions are directly harming the ecosystem.

It's definitely more nuanced. That said, I'd rather only have to fix memory unsafety in the language than potentially every program written with it. It reduces the audit surface area to just "unsafe" and Rust itself.

That data point doesn't say much before accounting for some confounding variables:

- the complexity of the code base

- the attack surface

- the amount of effort to find the CVEs

Sure - and i wouldn't put too much weight on my examples, i picked them as a direct refutation to the GP's claim "Well, empirically, no, that's not the case".

That said - 2 & 3 in your list seem like they'd lend further weight to Postifx rather than Rust in this scenario. Postfix is a network service forced to accept any and all input from remote attackers. Postfix has been around longer and has marketed itself as secure, that's meant it's attracted the attention of capable attackers keen to demonstrate their prowess.

I think the rust code base is more complex than postfix though. Postfix's architecture borders on a work of art.

Completely uncomparable. This is a great example of why CVE counts are a useless metric.

Large teams failed in part because they are large. I must point out djb succeeded. Empirically, it is possible to write secure programs in C.

djb didn't succeed with qmail. It had a security-sensitive overflow on 64-bit, and djb's excuse for why it doesn't count is unconvincing.

I’ve worked on C++ code bases that were stabilized to the point where crashes were usually hardware bugs.

I strongly prefer Rust these days, though it has poor support for certain advanced use cases.

Another example is Redis. Redis crashes are usually hardware bugs. This is why Redis includes a memory tester, which is run on crashes, so that it does not receive spurious bug reports.

https://github.com/redis/redis/blob/unstable/src/memtest.c

But as the article points out, it depends on the situation. The article is right that memory safety matters a lot more in certain areas than others.

The teams you refer to are working in areas where every exploit is a huge risk, places like a browser or an OS kernel. There is a reason Rust originated among people working on a browser. What you describe is correct for them. But the article has several examples of other applications where the tradeoffs are different.

Another example: LLVM is written in C++, but that's basically fine. A memory safety exploit in clang might be annoying, but it's not a vector for taking over people's machines like a browser is. A safety exploit in clang's output is where the real risks lie, but memory safety of clang itself doesn't help there.

Well, LLVM being written in C++ is fine for memory safety, but let's not kid ourselves. If LLVM were written in Rust it would have function level parallel compilation already. LLVM compilation is not parallelized yet because it is written in C++.

I don't know enough about why LLVM doesn't use parallel compilation, you might be right there.

But there are many C++ compiler codebases that do have it, such as: every major JS/Wasm VM (V8, SpiderMonkey, JSC), Binaryen, etc. Those do run in environments where parallel compilation matters a lot, unlike LLVM (which assumes parallelism in the build system, as you compile to object files).

Rust wants parallel LLVM, because Rust has larger compilation unit.

Interesting. That might be very specific to Rust, though. In that case, LLVM might not be the optimal backend for rustc.

But this isn't an issue for most LLVM languages, which either assume parallelism in the build system, or they get enough benefit from ThinLTO (which is the parallelism LLVM does support at the link stage).

>If LLVM were written in Rust it would have function level parallel compilation already

No one is stopping anyone from writing a compiler backend/optimizer in rust, yet it hasn't happened.

Cranelift exists and is in production. Hint: Cranelift does function level parallel compilation.

> LLVM is written in C++, but that's basically fine. A memory safety exploit in [LLVM] might be annoying, but it's not a vector for taking over people's machines like a browser is.

Hrm, is LLVM no longer used for shader compilation? I think most people wouldn't like it if compiling a shader could delete your home directory.

In such cases one can sandbox LLVM, either in another process or using wasm etc., and that has been common for many years. It's practical to do so because it's a pure computation with a clear interface (compile source code to machine code).

Yup, Mesa is no longer using LLVM except for CPU fallback.

I think this argument gets sillier every year, and passed the silliness event horizon probably at least a decade ago. Taking what the article says for convenience, memory safety prevents "buffer overflows, use after free, and use after return". My question is, what are you doing that actually depends on those things? Why would you consider them features?

There is of course a small handful of answers like accessing memory-mapped buffers for hardware access, but these are fractional percentage exceptions, not things to bend your language choice around. They're fractional percentage exceptions even in programs where that's the primary use case; you're still better off using an "unsafe" function to do your little bit of magic, then using the memory safety everywhere else.

And... it's not like C is a nice language. By modern standards, C is just terrible. Unpleasant. Poor affordances that lead to bad designs, where you end up giving up on the right answer because it's just so darned hard so let's just take some shortcuts. Spikes everywhere you walk, knives everywhere you touch. Missing basic features to any modern language like closures, and if you do bash them in they come with so many caveats that they're hardly recognizable.

If you are still someone who learned C 30 years ago and still think it's just somehow incomparably amazing and nothing can compare, please, spend some time with a couple of other system languages long enough to come to fluency, then re-evaluate. All the other languages have been polishing for decades now, and they've straight-up passed C for all language-related purposes. Yea verily these ~25 years ago when I started in this industry there were still cases to be made that C or the C++ of the time was still necessary to get the performance you needed, but those days have passed. The only remaining place I can understand using it are those fringes where it's just the only choice, and even then, the newer languages are marching along and if I were stuck in those industries I'd be trying to get on board as quickly as possible.

In my experience, having a large team makes it harder to achieve memory safety in 'MMM' languages. The ideal situation is a tractable codebase written by a small team of competent developers who can understand the entire scope. As you introduce more people, there are more potential points of failure

Do you have an argument why CHERI wouldn't work? As far as I know CHERI has not been tried yet.

CHERI doesn't do anything for use-after-free. It's basically a bounds checking solution.

Reply on news.ycombinator.com