Linus Torvalds on where Rust will fit into Linux
zdnet.com
zdnet.com
I am a C programmer by trade who has played with Rust. It is a great language but there is very little you can do safely. In higher level projects you can mostly push the un-safety into the standard library.
But when you have to do everything yourself, all of the "buggy" code has to be marked unsafe anyway. Allocating and accessing(!!) memory is unsafe. Interfacing with C or asm is unsafe. Your buffer/vector implementation is unsafe.
Basically, all the common culprits for memory bugs in C would be the code marked unsafe if rewritten in rust. So what is the benefit?
One of the key features Rust enables is that you can build safe abstractions on top of unsafe things. Another key feature is that those abstractions often mean compile-time, but not run-time, cost.
If you build the right abstractions, you can minimize the amount of unsafe code pretty dramatically. I don't have numbers handy for our codebase (and it's not open source yet, but will be eventually), but as an example, the Redox documentation (a Rust-based kernel that is, uh... linked... below) says: https://doc.redox-os.org/book/ch01-07-why-rust.html#unsafes
> A quick grep gives us some stats: the kernel has about 300 invocations of unsafe in about 16,000 lines of code overall. Every one of these is carefully audited to ensure correctness.
The advantage here is that, if and when there are bugs in this code, you only have to look at the modules containing those unsafe invocations, rather than the entire codebase.
There are also other advantages to the Rust language that make it worthwhile too, regardless of safety percentage, but it is true that this is often people's first concerns. The code that landed in linux-next has a lot of unsafe in it, because it is specifically trying to build out the proper abstractions here. The idea is that drivers built on top of this code will not need nearly as much, generally.
That is a brilliant feature I had not thought of. Thanks!
I would be cautious with statements like that. You can limit some kinds of operations to unsafe code, audit the daylights out of those, build abstractions that make it impossible to use that unsafe code from safe code in an unsafe manner, and still not be doing the right thing.
A recent example of using private constructors and zero-sized compile time invariant enforcement is an embedded blog here: https://www.ecorax.net/macro-bunker-1/
The embedded rust community uses this typestate pattern to build safe abstractions around pin interactions. Very similar things can be done to build safe abstractions around the volatile memory accesses etc that the kernel does.
Even extremely performance-oriented and low level projects such as the standard library or the Redox kernel are less than 5% unsafe code. There's lots you can do safely.
The benefit, and really the point of unsafe, is to encapsulate the unsafe stuff behind safe APIs and to use Rust's other features, like the borrow checker for example, to enforce correct usage of such API.
Ex.: I was using nanovg (vector graphics library in C) to build a GUI app some day. I was using Rust and a bare-bones FFI library that provided a 1:1 mapping to the C API. On top of this unsafe C-API, I built a wrapper library in safe Rust that doesn't allow me to use nanovg incorrectly. And not in the use-after-free sense of incorrect (that too), but instead in the "you can't call this function between these other two, that's undefined / will produce unexpected graphical results".
So the power of unsafe is to encapsulate "unsafe" behaviour and to enrich it with compile-time information, to build guarantees around it.
You've answered your own question.
> I'm interested in the project, but I think it's driven by people who are very excited about Rust, and I want to see how it actually then ends up working in practice.
(Deleted because I only meant to bring up the broader phenomenon of what makes a person feel "excited" or "unexcited" about a language; I was specifically trying (unsuccessfully) to avoid the hotbed topic that has arisen below)
EDIT: it's all good! I didn't think your comments were bad.
If we're talking about the former, I think the actual instances of this are few and far between, and are complained about far more than actually occurs. At one point I was actually trying to get hard data on this, but there really were only a handful of times I could find this seriously occurring, and they were pretty easily dismissed. The folks who talk about it like a plague of some kind never seem to show their evidence, or link to one or two examples at best.
If we're talking about the latter, it shouldn't be surprising that folks who like a language write software in that language. That's why languages exist. I don't know why some people really want to say that folks cannot write whatever software they want in whatever language they want.
This is probably depending on the environment. Corporate environments would move very slow, open source kind of fast and SMB/SME somewhere in the middle. All of them plagued by cargoculting, but especially average performing SMB/SME are the most guilty, and also where I've seen the most "Hey, we should probably rewrite this service in Rust because it'll be better for sure" since some years back.
Even if we assume these are all legit issues, there have been 66 total in the five years this repository has existed. That is a miniscule number.
If we actually look at the issues, many of them are not instances of an outsider demanding a project being rewritten in Rust. Let's look at the most recent five issues, to pick a random sample:
* https://github.com/ansuz/RIIR/issues/62 This is showing a branch that the team themselves made. It is not an outside demand to re-write it in Rust.
* https://github.com/ansuz/RIIR/issues/59 This one does appear to be legit. Not a ton of drama here, a polite conversation that was closed without issue, even though its existence is generally a bit annoying, yes.
* https://github.com/ansuz/RIIR/issues/58 This again is someone who is doing this for their own project.
* https://github.com/ansuz/RIIR/issues/57 This is a transcription of a joke in a telegram chat.
* https://github.com/ansuz/RIIR/issues/53 This is a person who opened an issue on this project, but does not appear to have asked upstream about it.
* https://github.com/ansuz/RIIR/issues/51 This one is amusing, given the article, but again, isn't an upstream request.
Let's look at recent closed issues:
* https://github.com/ansuz/RIIR/issues/67 This is a person who made this issue, then two minutes later, opened an issue being a jerk on the target repo. Yes, that is annoying for the person who had the issue opened, but is hardly an indication that these are real requests. This person is trolling.
The next seven most recent closed issues are someone who has just opened issues with random technologies they thought they'd be funny. I was only gonna do five, like the opened issues, but it's clear this is a series, so have a bonus two.
So already, we have like, let's round it up: two real issues here, out of fourteen. I am not convinced.
(Your hacker news link seems like nonsense to me?)
Not that it isn't a worthy effort, but it's still an effort by people who are more excited about Rust than they are acting in the interest of security.
Edit: It's almost a meme at this point that you can't even be remotely critical of Rust on this board without kneejerk mass downvoting.
I'm also well aware of lots of other security problems facing Linux that need love and the steps being taken by other operating systems where Linux lags behind.
So while I'm not saying that there isn't a concern about security by the people involved, I am certainly saying why this as opposed to other things.
I have my pet problems and areas where I prefer to work too.
Do tell, I am very interested?
> but it's still an effort by people who are more excited about Rust than they are acting in the interest of security
Needlessly dismissive. What if the people got excited because of the increased security?
If only people were so excited about making OpenSSL not a flaming hot bag of shit.
Can't wait to find out what that patch on Thursday will be for.
My 19 years of programming have taught me that something that continuously craps the bed (like OpenSSL) needs to be rewritten because from one point and on it's clear that the cost of maintenance exceeds the cost of rewriting.
Furthermore, do consider that some of us were programming C and C++ and found them sub-optimal. But here we would likely venture into language wars because there are a lot of programmers like myself out there who will never subscribe under the notion of "C is fine, you are just doing it wrong".
I would agree and it was already done by the OpenBSD project. Google started BoringSSL/Tink. Amazon created s2n. Linux continues to use OpenSSL. It's still probably in the top 3 problems with security in Linux that needs fixing.
The rest of your statement is just making my point for me.
Rants are fine and we all do them but I will strongly disagree with any responsibility or guilt transferred to me because I am excited about Rust and not about OpenSSL. As you yourself pointed out, alternatives exist but aren't adopted so the problem is political and not technical. Right?
> The rest of your statement is just making my point for me.
Not sure what you mean but okay. ¯\_(ツ)_/¯
> Not sure what you mean but okay. ¯\_(ツ)_/¯
Blame the carpenter, not the tools.
C or C++ isn't a real business problem domain that anyone has. Nobody's committing their budget to fix that. That's the kind of problem only held by ideologues and other unreasonable people.
My point was that people who care about fixing problems in the security domain will look to fix the biggest problems they can in the security domain. They won't look at it and say "no, I can't fix this because C/C++ is unacceptable to me."
That's someone who cares more about the tools than the problem domain. Whethor or not you think C/C++ are suitable to the problem domain of writing secure software, software will be written in C/C++ and must be secured.
Turn down work all you want though. More for the rest of us.
Sometimes that's true, sometimes it isn't. It's a cost-benefit analysis like almost anything else.
Everyone is free to disagree on this -- or like you, misinterpret my motivation as "caring more about the tools than the problem domain". Quite the contrary, I want the problems solved but I also put deadlines and energy budgets on trying to solve problem X with tool Y and if it doesn't cut it within the deadline and/or the budget then it's out. Sure, that means more C/C++ work for you. I am OK with that.
But let's not pretend that OpenSSL can only be written in C or C++ now. We all know that's not true.
> Blame the carpenter, not the tools.
It's no accident that professional craftsmen utilize innovation in their tooling as well, like Sushi chefs using ceramic knives for some fish and steel knives for others. Ceramic knives weren't that popular some mere decades ago.
Your black and white philosophy makes no favours to your argument.
It seems you take an issue with the rising popularity of Rust. But let me ask you this: if you are so confident in C/C++'s superiority, why are you bothered?
I don't have a problem with Rust at all. It's a fine tool and I'd be happy to use it (and have), just like any other. I work on interesting problems and am tools-agnostic (aside from certain languages being suited or not to specific domains).
I have a problem with people who lose their objectivity as a result of their enthusiasm for Rust and go around shouting in every project like it's the most important thing to happen in the last 30 years of Computer Science. It wouldn't be such a meme if people didn't actually friggin' do this.
I hear this a lot and I've never witnessed it once. ;)
It's all filter bubbles, dude. Chill. The world is not ending.
As an example: the comments from both Linus/Greg and the Rust community that relate to the OP are quite balanced and nobody is shouting for revolutions.
---
...That being said, I wouldn't make grandiose claims about Rust myself even if it solved a lot of problems for me, but still: Rust is an important innovation regardless. It seems a lot of the software area has chased its own tail before we had some languages step forward and solve various problems that people kept trying to fix themselves on an ad-hoc basis (and kept failing at it).
Rust is one example, Erlang/Elixir are another, and when OCaml finally gains parallel abilities, it will be an additional (and very fine) example as well. Surely other examples exist.
If we're now discussing just the point you quoted me on -- I on my side take issue with the people who are way too liberal and love to pretend that it doesn't matter what language is being used regardless of the business and technical problems. Because oh my god yes, yes it does matter. A lot.
Rust is already getting foot-guns like C++ did.
Modern C++ (>C++17) with RAII and neat libraries is already an elegant, high performance and high-productivity language with good IDE/editor support.
And its not like C++ is standing still. You have standardised concepts, modules and co-routines out this year. Modules will fix several of C++'s shortcomings wrt build tooling and setup. I am personally looking forward to C++ 20 co-routine support in all the major compilers - will enable one to develop graceful-async, portable and stable libraries without all that Rust churn needing re-writes all the time.
Competition is good, but its best to take the Rust blinkers off a bit.
I am not fangirling for Rust at all.
I am saying that it's much easier to make costly mistakes in C and legacy C++. I am aware of the modern efforts in C++ but they don't help with the literal mountains of legacy code that has to be maintained out there...
Of course you have to care about the problem domain first, but it's absolutely wrong that nobody commit their budget to fix C++ safety problems. There is a massive amount of static and dynamic tooling to mitigate its problems and a substantial part of the C++ evolution these last years was to improve safety. Rust was able to go further than C++ in this area since it is not yet constrained by decades of history.
That's a weird way to put it. I would blame the carpenter for picking the wrong tools.
BTW, rustls exists and is a rewrite of OpenSSL of sorts.
I do wonder how it happens as I upvote such people, but I've been posting here for years and still don't have downvote privileges. I do wonder sometimes who controls the downvotes.
I also think that's why lisp people are so feverish about lisp as well: as you get more involved in the language with its charms, the more excitement you get and you want to see the language in more places/contexts
I'm interested in what Torvalds meant by these odd header files, does anyone know?
/*
* This returns a constant expressionn while determining if an argument is
* a constant expression, most importantly without evaluating the argument.
* Glory to Martin Uecker <Martin.Uecker@med.uni-goettingen.de>
*/
#define __is_constexpr(x) \
(sizeof(int) == sizeof(*(8 ? ((void *)((long)(x) * 0l)) : (int *)8)))"- this will break the minds of everybody who ever sees that expression."
It indeed broke my mind!
sizeof((void*)1)sizeof(*(void*)1)
I'm not a C dev BTW.
Edit: I am also a proponent of careful re-writes of very old userspace utilities (eg: ping) accompanied by very rigorous tests to ensure there aren't any behavour changes
That would be great, but a prerequisite for that is that the rust language is stable to ensure that there aren't any behaviour changes in the compiler itself.
That being said, it is true that Rust doesn't yet support all the platforms that are important to the kernel. That's one of the reasons this is starting with drivers, and is one of the reasons why some folks are pursuing a Rust frontend for gcc.
If you're still running a system with an arcane ISA, support it yourself. 99% of the people running modern Linux are interested in three CPU architectures (four or five once RISC-V gains traction), all of them supported by Rust. Supporting old or obscure ISAs is not a priority at this point compared to bringing kernel memory safety into the 21st century, which Rust helps a lot with.
A far as I know, the latter supports many architectures the former does not.
It's really interesting that any C code already has to have a memory management ownership model, and the consistency of that model makes Rustification easy / hard.
For example, an H264 encoder/decoder module in the kernel: let's say I have a Freescale/NXP i.MX6 platform with a HW H264 decoder, and I have to create kernel module to talk to it. Then use this module from an user space application to decode a video file.
What I did (modified actually) in the past is a module that allocated a couple of (aligned) buffers in video memory and then passed them around between the HW decoder, kernel module and user space application.
This is usually done to not force a deep copy of >100KB buffer for every decoded frame. This memory is shared between different parts of the implementation, meaning that the same buffer belongs to up to 3 parties at any given time. Also, the allocation is done once and lives for the entire application life, usually allocating like 3 buffer so when one buffer is being decoded I can still use already decoded buffers for writing them into a file.
I know there is an alternative (hack?) using something called Pin (possibly clashing with, like, a GPIO Pin?) but I didn't get so deep already with Rust to really understand it.
Ah! whatever driver/module they do, it should be accessible with C. So Rust must fill the gap between "Some C code in userland" <--> Rust <--> "More C code in the kernel", so I'm really interested to see how ownership works in this case.
Userland C app says: "Hey Rust kernel module, take this struct and do something", the I go and change the contents of that struct from another thread in my userland application. Would this be possible?