Curl audit: How a joke led to significant findings
blog.trailofbits.com
blog.trailofbits.com
[2023-02-14] <The article> the fuzzer quickly uncovered memory corruption bugs, specifically use-after-free issues, double-free issues, and memory leaks
Rust was never expected to be a magic fix-all, but after the C++ committee‘s stubborn refusal of introducing some sense of memory safety into the language in favor of keeping ABI compatibility, it might turn out to be a self fulfilling prophecy.
Yet everyone’s idea of safety still seems to be “just write bug-free C code, bro”.
It seems like "write memory safe C" was the best technique in 1996 if you wanted any semblance of portability/performance.
... still have multiple easily-uncovered memory issues.
Nobody writes safe C. Believing otherwise is delusional. This has been known for decades. But many believers still exist.
This is factually incorrect. You can write safe C code which is even correct according to some specification, as in "guaranteed bug free". This is not what most people do (or even know how to do) though.
There are lots of specification driven C applications, which rely on (1) the specification being bug free, (2) the application of the specification being bug free, and (3) the model or proof assistant for the specification being sound and bug free. Getting all three of those is historically very difficult.
It's a straw-man that's trotted out every time, and it's utterly meaningless beyond academic arguments. Even if it is true, it's not relevant for any practical purposes.
And I'll point to CVE-2015-4041 as evidence in my favor - that's a buffer overflow: https://www.cvedetails.com/vulnerability-list/vendor_id-72/p...
For all the code-golf insanity that Perl supports, it's an extremely effective language with a truly stunning degree of both flexibility and performance.
Unfortunately it still lacks meaningful marketshare though.
Meanwhile you can just use `parallel` in your shell scripts with very little ceremony.
Lua is great in many ways but looking at it in 2023, it's kind of stuck in the past.
Common Lisp, depending on how you define memory safety.
If you remove the 'still exist today with meaningful market share', in '96 you had more options (e.g. Modula-2/2+/3 family).
There were others.
https://www.modula2.org/reference/isomodules/isomodule.php?m...
It also has pointer arithmetic, casting and sizeof in a SYSTEM module:
https://www.modula2.org/reference/isomodules/isomodule.php?m...
I'd actually like to see a Modula-2 example of, say, dynamically allocating an array object, which is dynamically sized. Because obviously Modula has better arrays which know their size, which a C program has to express all by itself with size fields and pointers.
I worked with Modula-2 quite a lot around 1990 but didn't do anything of the sort. I didn't know anything about any ISO standard or memory allocation functions provided thereby. A cursor search seems to indicate that it didn't exist until 1996; basically years after "peak Modula".
Does it though? I think "the industry" has pretty much come around on C being unsafe, and generally not a great choice of language for new projects.
But the fact is that there's a lot of C code out there, and we can't just ditch or rewrite it all overnight. In an ideal world, we wouldn't be starting from here. But we are, and it's going to take time to get to where we need to be.
Heck, take a look at the recent story about "Considering C99 for curl": https://news.ycombinator.com/item?id=33704054
From the linked article:
> The slowest of the “big compilers” to adopt C99 was the Microsoft Visual C++ compiler, which did not adopt it properly until 2015 and added more compliance in 2019. A large number of our users/developers are still stuck on older MSVC versions so not even all users of this compiler suite can build C99 programs even today, in late 2022.
What are the chances that the kinds of people who are still doing development on systems where the C compiler is from 2015 or earlier, or on a more niche system that Windows that lots of new safe languages haven't been ported to yet (and such ports aren't even on the radar), are going to able to start moving to developing in rust or whatever anytime soon?
If you look at some of the Rust threads on HN right now [0], you find many commenters claiming that there is no reason to replace C with Rust. It could be a vocal minority, but there's still a sense that "not everyone is on board".
If you have some reason to choose C today, I'm not sure Rust is a contender to stand in as a replacement. They are vastly different languages. That doesn't mean there is rejection that C's memory model is unsafe, just that there isn't a "better C" recognized to choose from.
Because otherwise, you could write Rust 2018 edition (for example), and there's nothing preventing a situation where your currently-valid Rust-2018-edition program stops compiling (or behaves differently) on a future version of rustc even when compiling to that specific edition, just because some behavior is (in the future) considered a compiler bug or something. And it would be perfectly valid to cause such breakage, because there's no definition of "what is Rust", other than "what this specific compiler in this specific version accepts".
I'm not talking about whether such possible backwards-incompatible change on a Rust edition would be good or bad. Just that with the current monoculture around a single implementation, there's not many guarantees.
A specification, or multiple implementations, would at least add some (good) friction.
I'm not denying its current value as a safe language; just that, as it is now, I don't consider it as a good choice for programs that must stay alive for decades, like `curl` or an operating system.
Unsound behavior is considered a compiler bug, and is liable to change if you accidentally rely on it. I would hope that having multiple compilers won't change that. The motivation to actually fix soundness holes is one of the primary differentiators of Rust.
That said, these are very rare.
I'm looking forward to Ferrocene's specification and the gccrs implementation.
Many of the people writing code in C do not want to use C++.
And many of the embedded use cases for C already involve very careful memory usage. If you've banned malloc and don't have threads, you've removed a huge swath of common bug sources right off the bat.
For someone writing code to setup registers on a DMA controller, Rust has nearly 0 benefit.
(That said, no one should be writing new command line utilities in C, parsing arbitrary data in C is dangerous!)
This thinking is understandable but it’s basically pattern matching. There were two successful languages that followed this model and so a language that seeks to replace them should adopt the same model of design by committee followed by multiple independent implementations.
But there’s no reason to think that a language needs multiple implementations to succeed. Python has had one implementation used by all its user and that hasn’t prevented from becoming a top 3 language by usage. Ditto with other very popular languages like Go, Ruby, TypeScript, C#.
I don’t understand the concern that your code will stop compiling with a future version of the compiler. Firstly, Rust hasn’t broken backwards compatibility in 8 years and doesn’t plan to in future. Secondly, why do you need to upgrade the compiler? If it works on the current version you can continue to use that in perpetuity.
Honestly, having multiple implementations would be a curse. I see nothing but negatives. Even the smallest proposals (like embedding data in a binary) take a decade to get buy in and 3-4 years to be implemented. Convincing everyone of the benefit of a feature is just too difficult so a lot of good ideas are never put forward, let alone implemented.
And the difficulty of getting it working across compilers and operating systems. How hard is it to write C++20 code that will compile with all major compilers on all major OSs? On Rust it’s trivial. To put it simply - it’s trivial to answer the question “is this Rust code valid?” If rustc compiles it, it’s valid today and will always be valid (modulo bugs).
You want alternate implementations? There’s a WIP - gcc-rs. But thankfully they haven’t forked the semantics of the language. It should and shouldn’t compile all code that rustc does and doesn’t. Otherwise it’s a bug in gcc-rs. Anything else would make the life of Rust users painful.
In summary, multiple implementations is a bad idea for many reasons. If you can express the benefit beyond “that’s the way C and C++ do it”, I’m eager to listen. But please explain how Python and Go managed to succeed with one impl.
By creating a standard, and allowing multiple implementations, someone creating a new OS or a new CPU architecture can create their own implementation of the language without having to wait for anyone else to deign to create the port.
Maybe a new implementation won't compile very fast, or produce great code, or it might not even catch all the incorrect constructs that it ought to. But if it can make all the existing code written in that language suddenly available for the new platform, that platform instantly becomes orders of magnitude more useful and powerful.
> How hard is it to write C++20 code that will compile with all major compilers on all major OSs?
That depends on the code. You want to compile Firefox? You might be out of luck.
You want to compile curl? Based on how portable it already is, chances are it might work out of the box once you've got a libc implementation working.
You want to compile Gnu bash and coreutils, to get a basic Posix shell environment up and running? It might be a bit of work, but easier than having to reimplement a whole userland from scratch.
> But please explain how Python and Go managed to succeed with one impl.
It's not that the language can't succeed - at least, not on popular platforms that the devs are interested in targetting. It's that those devs get to dictate which platforms are even capable of making use of the software written in those languages.
And that just rubs me the wrong way.
Why does that require multiple implementations? There's nothing stopping said party from adding support for their OS/arch to rustc directly today. There are multiple examples of this already: fuchsia, Sony PSP, Nintendo Switch, etc [1]. Now if you want support in rustc itself out-of-the-box this does require LLVM support your target as well. But even then, again nothing stopping you from adding LLVM support too. The avr-rust project for instance maintained an LLVM/rustc fork for a while before those patches were upstreamed.
[1] See the wide variety of targets with varying support today: https://doc.rust-lang.org/nightly/rustc/platform-support.htm...
Or maybe you don't want your work to be redistributed under the original project's license, and would prefer to have a copyleft/permissive/proprietary licensed version instead, which can still call itself a "real" implementation. (e.g. mono or gcj/openjdk. Or gcc, originally.)
Adding other tiers is welcomed. Take a look at the supported platforms (https://doc.rust-lang.org/nightly/rustc/platform-support.htm...) and see if there's a platform you'd like to target that isn't a tier 3 target at least. There's HaikuOS, PlayStation1 etc. I'm on a Tier 2 platform myself and I'm really happy.
If you'd like even more platforms, there's an ongoing project to add a GCC backend to rustc (https://blog.antoyo.xyz/rustc_codegen_gcc-progress-report-20).
> if it can make all the existing code written in that language suddenly available for the new platform, that platform instantly becomes orders of magnitude more useful and powerful.
I agree with this, but creating a front end that can actually compile all existing code correctly is a massive task. If the compiler is modular enough, you should be able to contribute just a backend, rather than reimplementing a frontend. I'm strongly against only the frontend reimplementation, because it's no longer clear what "valid" Rust code is.
In summary - I completely agree with the importance of targeting niche platforms. I think supporting multiple backends (LLVM now, libgccjit and Cranelift in future) gets us there without fracturing the ecosystem.
> but easier than having to reimplement a whole userland from scratch.
Probably. But once it's reimplemented (https://github.com/uutils/coreutils) I think it's really cool you can run get binaries for Linux, macOS and Windows with one build command.
I won't deny it's pattern matching, but it has less to do with what C did, and more with what history has been. I've come to distrust too much centralization on important things.
Take Node.js as an example. It started controlled by a single company, then thanks to the community a Node.js Foundation was formed with an open governance model. But now that Node.js Foundation no longer exists, instead it's controlled by the OpenJS foundation, which is basically controlled by big corp.
I don't want something similar to happen to Rust.
That said, I don't mind Rust as it is today for user applications. And I happily use languages that have a BDFL, that are even less sound as Rust, so the problem is not Rust's success (this can't be denied).
But I honestly think that if Rust wants to replace C (or even SPARK) on critical systems, and even make universities eager to teach it, it would be easier to play by the industry's rules, than to offer a "black box" ("what this compiler accepts"), for the lack of a better term.
Also, as I pointed out in a different message (you might have missed it), I'm aware of gccrs, and I'm looking forward to it reaching feature parity with rustc; and I also appreciate the effort being put by the Ferrocene[1] people, because I think it with Rust getting wider adoption in the industry.
Those are just people annoyed that their knowledge is getting outdated, and grasping for any medium-approved excuse to not learn the new thing. Those believe that Rust will replace C, they just don't like it.
(Anyway, C, like COBOL, won't just die. There's no reason to be afraid.)
Yeah but do they like the language tho.
I actually don't see many claim that in that thread. There are 196 comments and I didn't read all of them, but only one greyed-out top-level comment says so, and what I can skim from the other comments I don't really see a whole lot of people say "C is fine!"
There are some concerns about performance, and some other comments about (alleged) "Rust spam", but that's not the same as "C is fine!"
In my experience "C is unsafe" was already generally agreed on about 20 years ago, which is one reason many choose Java, Python, PHP, etc. at the time. It wasn't a universal consensus, but if you were to take a poll in 2003 I bet it would be well over 50%.
Seriously I can't understand the threat model here.
I think the realistic attack model here is:
1. SoHo router that has a public accessible endpoint that takes inputs boiling down to "safe" cURL flags;
2. Those flags can actually induce memory corruption;
3. An attacker is able to execute arbitrary remote code in the context of the cURL subprocess through a payload packed into the "safe" flag.