Linux kernel drivers in Rust might become an option in the future
lwn.net
lwn.net
Check out the demo in PR #122, which lets you create three boolean sysctls and a character device that prints the state of those sysctls in JSON (using serde).
We gave a talk about it last week at Linux Security Summit, I'll submit it once the recording is up :) Slides are at https://ldpreload.com/p/kernel-modules-in-rust-lssna2019.pdf .
Though it would probably make sense to make the API more "Rust like" and avoid things like the multiple cstr!()
Some things are definitely trickier than others, for example how to deal with the different options of kmalloc if it's rust that's allocating memory
I think there's work upstream on custom / multiple allocators; we'll plug the various kmalloc(GFP_FOO) flags into that once it exists. It is super beneficial to use the standard library's collections and third-party crates that use them.
It’s to auto create rust bindings. Link between rust crates and kernel build system
For instance, the kernel wants you to define a character device by passing a struct with a bunch of C function pointers for how to read, write, seek, ioctl, etc. on the device. Rust has a trait / interface system that's a good match for this use case, so in src/chrdev.rs we define a Rust trait with all these methods, and we have a helper function to create the C struct with FFI-safe function pointers that call the various methods on the Rust trait.
The broad goal is that you don't need to use the "unsafe" keyword to write kernel modules that don't access memory directly themselves (filesystems, network protocols, etc.: device drivers might still need unsafe code where actually talking to the device, but it should be as little as possible). That means the interface used by kernel modules can't involve any unwrapped C functions or raw C pointers.
eBPF's approach is to generate code that's verified to be safe and that's limited in the operations it's allowed to do in the kernel, using trusted helper functions to access kernel data structures and functions. I couldn't help but draw an analogy to unsafe wrappers for Rust.
In its current form I might not want to write core logic of a device driver in eBPF, as it started out as something suited for mandatory access control policy, packet filtering, and auditing. That's changing though, and people seem to be demanding more and more functionality in eBPF. I'm admittedly bad at predicting the future, but one thing I can say with some confidence is that eBPF's capabilities are going to increase with time.
I would also be hesitant to attempt to write a device driver in Rust because of the degree to which I'd have to interact with other subsystems and structures of the kernel that are still written in unsafe C. I wouldn't have an intuition for the incremental benefit of using Rust for some of the core logic of the driver while still having to use unsafe wrappers to muck with all the other parts of the kernel where, if I get it wrong, my driver can still oops/hang/etc. Would the complexity of mixing two languages with unsafe wrappers plus increased code size "pay for" any incremental benefit, or would I have to expect future dividends from eventually having "enough" of the kernel written in Rust for it to be a worthwhile investment?
One advantage of eBPF, I suppose, is that you can write your code in old familiar C. With forthcoming support for bounded loops, I expect people are going to be proposing more and more use cases for it. I can imagine scenarios where one camp will say, "This is a job for Rust," and the other camp will retort, "Actually, this job can currently be done in eBPF." eBPF having the other distinct advantage of already being a supported feature of the upstream kernel.
This is all pretty new to me, and so all I have are my impressions and intuitions. I'd appreciate hearing more perspectives on this.
https://kth.diva-portal.org/smash/get/diva2:1238890/FULLTEXT...
I mean, I personally think Rust is cool and all that, but come on, who does that?
In any case, FF shows that rewriting a complex C/C++ codebase in Rust bit by bit is feasible, though again, that doesn't seem to be the intent in this case.
There are many more kinds of errors that people could make besides memory errors. Linux powers a good portion of the world, the vetting of kernel contributors should be done accordingly and the processes put in place should be strong enough that even if 'young and new blood' makes contributions that these are reviewed with a keen eye to all the lessons learned over the years, something those new and young people will still need to do.
I'm all for including newcomers into important open source projects but the kernel is the one place where I would expect some experience to be a requirement before contributing simply to avoid overloading the people further up the chain with a stream of obvious errors.
There is good precedent to warrant such vigilance:
https://www.newscientist.com/article/dn24165-how-nsa-weakens...
Accidents can never be ruled out but a malicious operator will have far less chance of getting away with something when detected early.
Linux is now mission critical, which means the rules of 1991 no longer apply.
Even if you want people to use more Rust, that kind of aggressive evangelism doesn't serve you well, and people find it off-putting.
Some people have an LDS complex (they've found their messiah and want to spread the word). Apparently there's also people who've decided that over-the-top inane "evangelisation" was a good troll and way to turn people off. A significant number of PL threads on /r/programming have a comment attempting this sort of shit-stirring.
AKA Mormon Church, who are pretty famous for their door-to-door proselytisation.
Or, some craftspeople/artisans prefer tools that are a joy for them to use instead of just a means to an end, and have the luxury to choose what they work with.
I was recently having someone do this on a web project of mine that's written entirely in ES6 Javascript
Yesterday Go, today Rust.
I'm also guilty of thinking language X is the next big thing, and that I should use it for everything. And that's happened more than a few times: Smalltalk, Eiffel, Python, Haskell, Lua, Go.
So having said all that, I really do think that Rust is the next awesome thing. Like the others on the above list, Rust has certainly stretched my brain with new and better programming concepts. And I think it offers some solid benefits that few other programming languages are providing right now. Its also interesting to start to see other languages like Swift incorporate concepts from Rust.
I even started working on Java applets for an educational website. The idea was to teach algebra / calculus concepts, and use a Java applet to interactively plot graphs and such.
Even at the time, I did wonder about the standard types vs. Objects split, though I wasn't to become an OO purist (temporarily) until learning more Eiffel.
Now what disappointed me was that languages like it, namely Oberon variants, Modula-3, Eiffel, Sather, all had a mix of AOT/JIT toolchain, with value types and low level capabilities.
Java nuked all of this, and is now trying to catch back those features that should have been there since the beginning (AOT kind of was, but only on commercial JDKs).
I have high hopes for graal native compilation and project loom.
(The concrete budget of innovation points varies per project. A government project will have way fewer innovation points than a weekend side-project.)
"That's not a new phenomenon at all. We've had the system people who used Modula-2 or Ada, and I have to say Rust looks a lot better than either of those two disasters.
I'm not convinced about Rust for an OS kernel (there's a lot more to system programming than the kernel, though), but at the same time there is no question that C has a lot of limitations.
...I don't think you actually solve any of the really hard kernel problems with your choice of programming language. The big problems tend to be about hardware support (all those drivers, all the odd details about different platforms, all the subtleties in memory management and resource accounting), and anybody who thinks that the choice of language simplifies those things a lot is likely to be very disappointed."
I disagree with him about Ada, for what it's worth, but the overall point is correct. The real problems of kernel development aren't things a borrow checker will help with. A big part of developing a kernel takes place before management of heap memory is even relevant. All this being said. For user-space applications, Rust has a lot to offer.
If I also need clang and rust for all the platforms, and learn rust, for some questionable benefits... That would make things hard.
I can see the arguments against needing 2 different C-compiler toolchains, not to mention how this may limit target platform-support to the minimum subset supported by both compilers...
But to argue that Rust only provides "questionable benefits" is really not reasonable.
Even the hipster Javascript crowd has discovered that providing more information to a/the compiler (Typescript) almost unconditionally provides higher code-quality and better results.
When will the C-crowd do the same? When will they shed their "I know better than any machine"-like elitism?
In other words, Rust is just another tool. Regard it as one without subjective emotions either way. Be constructive, don't only look for faults, but also ways how to overcome them. But don't close your eyes from them either. Acknowledging weaknesses is the first step to improvement.
Some possible questions and measures to consider below. I'm sure there's a lot more to add on this list.
1) Stability. Other than for development, unstable kernels are a no go. Can possible negative effects be mitigated?
2) Security is often what Rust is expected to bring on the table. So is Rust actually more secure in the environment kernel requires? This could be tested by "clean-room" reimplementing something that is historically known to have many security issues.
3) Are there showstoppers for kernel builds? Interoperability, build performance, architectures unsupported by Rust, etc. If so, could these be mitigated? Conversely, is there something positive Rust could provide?
4) How does it affect runtime performance? Average case. Bloat issues? Any pathologic cases? Any benefits?
5) What other unexpected it brings on the plate? Both benefits and disadvantages. For example, could Rust types also be used to catch errors other than memory related, like invalid states?
6) <Your consideration here or above>
This is nothing away from BetterC and other languages. If anything, this could open up ways to using languages other than C in kernel development.
That said, I don't think there's room for more than maybe two languages for the critical kernel components even in the long run. Not because of technical limitations, but human ones.
AFAIK neither Ada nor BetterC prevent memory safety bugs like Rust does... for example it appears neither of them prevent use-after-free of dynamic heap allocations, a pretty common sort of exploitable memory safety bug.
Borrow checking is done statically.
In other words, Box<> is just the equivalent of std::unique_ptr in C++.
The point here is that you can implement many kinds of complex and useful heap-allocated data structures in safe Rust code, without any reference counting, and the compiler will verify that you have no use-after-free bugs, or any other kind of memory-safety bug. The same is not true of (pre-SPARK) Ada, or C, or C++, or BetterC.
It is still WIP, yet to be deployed at large, naturally doesn't cover binary libraries, but it already a very good improvement.
Even if Rust would fail at large scale mainstream adoption, the fact that the community has pickup up Cyclon and ATS ideas to the point that Swift, OCaml, Haskell, Ada, C++, D, Chapel, ParaSail and eventually other communities started to adopt similar ideas, that alone is a major victory to everyone involved in Rust.
Maybe now with Microsoft having some care for Rust, the tooling situation will improve, but right now, even if handicapped, C++ lifetimes are easier to sell in some shops than rebooting the whole stack in Rust.
And in the end they all share the same goal, improved safety in our daily stacks, even the bits we only touch as user.
I think what you're trying to get at is that the borrow checker is based on static lifetimes (that's not the same thing as a "stack-based analysis", by the way). That's true. But that's simply because most lifetimes in practice follow simple static patterns. This is the same observation that motivates RAII in C++. Because heap management tends to follow the same few patterns over and over, judicious use of Rust compiler features and standard libraries can eliminate a lot of problems.
Crossbeam uses "epoch-based memory reclamation," a strategy for maintaining an object shared across threads without either locks or a global garbage collector. The tl;dr of the strategy is that there's a concept of "epochs," and if you update an object, you have to keep the old copy of the object around until the epoch is over. How long an epoch is alive is determined by how slow your readers are.
So, to access some data, you create a Guard object and pass a reference to that Guard object into the functions that access Crossbeam-protected objects. You then get a reference whose lifetime is bounded by the lifetime of your Guard object. (Typically you're going to create the Guard object in your local stack frame, but nobody's stopping you from putting a Guard object on the heap, getting an arbitrarily-long reference to an object, and blocking reclamation for arbitrarily long, if you really want.) You can safely access the data through this reference until its lifetime is over, and other threads won't reclaim the data (i.e., deallocate it from the heap) until your Guard object is gone.
The implementation uses unsafe, but that's not surprising, the implementation of Box itself uses unsafe so it can call malloc/free or whatever your platform equivalent is. What's important is that the safe interface can translate the requirements on paper into requirements that the borrow checker can check, using this Guard object.
std::unique_ptr can't do that. (Also more generally, std::unique_ptr isn't memory-safe - see the example in https://alexgaynor.net/2019/apr/21/modern-c++-wont-save-us/ .) There's no way in C++ to say "Here's a reference to some data in this unique_ptr, and by the way it's perfectly safe to have more than one reference to it, but you can't hold onto this reference forever because I'd like to deallocate it soon."
We're already finding that the Crossbeam-style guard pattern is helping us express kernel RCU in ways that are safer than what could be done in C. Namely, there are functions that expect to be called inside an RCU read-side critical section but have no way of enforcing that in C other than by adding runtime checks for the current state of RCU. In Rust we can pass the Guard object around and ensure that readers a) create a critical section and b) don't continue to dereference data once they've declared the end of their critical section.
Now, an environment without a GC will use ref counting sometimes but it will regardless of C, Rust, or C++
- can't deal with circular references without user intervention
- storing the reference counter is hard: either it clobbers a whole cache line or it ruins memory alignment
- atomic reference counting has the potential to utterly ruin runtime performance unless it used extremely carefully
And when one cares about performance, creating/destructing hundreds of instances per frame like on reactive approaches, it isn't the best use of CPU cycles.
If someone wants to make the case that SPARK would be a better approach than Rust to writing safe code in the Linux kernel, they should go ahead, but I haven't seen anyone make that case.
> If you now feel like using them, a preview is available in the community 2019 edition of GNAT+SPARK.
"C++ is really mature and there are so many C++ projects and developers!" "Yeah but it has these problems..." "Just use C++17, it solves all that!" ... but C++17 is not the language that is really mature and that all those C++ projects and developers are using.
> We've had the system people who used Modula-2 or Ada, and I have to say Rust looks a lot better than either of those two disasters.
From: https://www.infoworld.com/article/3109150/linux-at-25-linus-...
Linus has learned that anything he says is potentially going to quoted and quoted over and over again. So it seems like he only talks in extremes now. I always tend to look at it like he likes rust more than the other two and nothing more.
If he said the other 2 are alright, people would run with that quote, and vice versa.
"Would C++ fare better or worse than Rust on these fronts?" is the question I would have.
The one thing I will say about C++ from long ago was that you needed to be skilled at C++. You needed to understand it and you needed to know how to write code that wasn't going to cause you problems in the future. You also needed to work on a team that allowed you to use your skill. The language is so different now that it's practically I different language so I don't know how much of that still remains, but I guess quite a lot.
I've been doing some Rust recently. I like Rust. There are bits I think still need work, but generally it's a fine language. The biggest difference I see between it and C++ from long ago is that Rust helps you a lot. It's very convenient and friendly. It also scolds me when I do something stupid, which I appreciate.
However, I don't really feel like I need to know less than I did as a C++ programmer. With Rust, it helps you, but if you start causing problems for yourself you can be in for a world of hurt trying to understand what the compiler doesn't like. It will catch that thing that slips by you, which is awesome, but you still have to understand what's going on under the hood.
So in my book, it feels very much like C++ with a super friendly and super powerful linter. I mean it's really, really nice, but I don't think you could just hire random people off the street and give them a Rust compiler and say "This will protect you".
So to answer your question (finally), I think you won't really fare much better or worse, but the journey seems a little bit more relaxed with Rust.
I wonder if easing the maintenance burden of beleaguered open source maintainers is an even larger benefit in practice than catching errors at compile time.
Having a really sophisticated type system means...dealing with types a lot. I love Rust - especially ADTs and everything being an expression - but I wouldn't call it more productive than C++. And I think that was mikekchar's point?
See the 6th example here: https://doc.rust-lang.org/std/iter/trait.Iterator.html#examp...
I think that's probably an acceptable trade-off for certain types of development, and kernel engineering is probably exemplary of that type of development. I might be more hesitant to endorse it if it was the only way (or only way without major hurdles) since that would raise the bar of entry for people wanting to figure out how to make a Linux kernel module (the "make your own kernel module" howto's are empowering for Linux), but as an additional supported interface with the constraints mentioned in the article I think it makes sense.
Exceptions are incompatible with environments like the Linux kernel. This then makes you incompatible with the stdlib and most third party libraries. It feels like most exception-less C++ libraries also handle error handling in their own way. This means you are in this weird language+library word disconnected from using existing C++ code. You also have to deal with training, not just for kernel + language but kernel + language + special language variant with all of the gotchas of all three.
Herb Sutter is working on improving this situation but I feel it is a ways out and last I looked at it, I was a little uncomfortable with it.
Even once that is handled, there is the issue of what subset of the stdlib is safe to use. Rust has done some work to help with this with nostd and most third party code that isn't compatible with nostd could probably be easily fixed to do so.
bcraig is working on "freestanding" proposals to fix this in C++.
https://github.com/protocolbuffers/protobuf/blob/master/src/...
Let's say your object is going to contain another object that wraps a file, but constructing that file wrapper fails because the file doesn't exist. You are forced to construct your object, so you are forced to have something to put there. Maybe it's a dummy, maybe you can 'null it in some way', but it's immediately not as ergonomic. You also then have to either rely on the caller to not use the failed constructed object, or put a check in every method call checking for 'valid'.
However, I imagine it would be a huge undertaking to restructure Kernel code in a way that the rust compiler finds palatable.
C++ RAII would help a great deal, but Rust's main feature is that it's RAII on steroids.
It's not. Rust is being used for production systems by several companies, so it hardly counts as a "research language".
> Anyone trying to fix/debug things in the kernel now has to learn Rust.
Wrong again. If a driver for some SD card reader is written in Rust, then you won't come in contact with this if you're working on the IPv6 subsystem.
I don't know that Rust has ever been a research language in the first place, maybe they mean "in development", as in pre-stability?
The 2005-ish "one man side project" and "searching for its niche" phases do sound researchy.
Interesting quote: "Funny enough, the zero cost obsession really wasn't me. I was ok paying some runtime costs. We just got overrun by C++ folks :P"
I agree that there's a cost to introducing a new language to a codebase. On the other hand, there's a cost to being stuck with only C, which manifests as security vulnerabilities and high cognitive overhead for want of better abstractions. I believe the Linux maintainers are taking a sensible and conservative stance here in allowing use of Rust, but not allowing Linux to depend on it.
I would accept that. The idea that C++ code from ten years ago can't always interoperate with C++ code written today is indeed a problem, but I think it's a very different problem that you're running into if you're trying to use Rust applications and libraries built ten years ago.
> there's a cost to being stuck with only C, which manifests as security vulnerabilities and high cognitive overhead for want of better abstractions
There's also a cost with abstractions. People who use Rust thinking it has "zero-cost abstractions" should probably get help crossing the street as well.
> I believe the Linux maintainers are taking a sensible and conservative stance here in allowing use of Rust, but not allowing Linux to depend on it.
100% agreed. Rust may indeed be the future, but the only way we'll know for sure is if we try it.
Hardly any different.
We actually have an objective experiment to answer this question: the rust-afl trophy case [1]. It is remarkable how different it is from the upstream AFL trophy case, which tests C and C++ code [2]. An enormous fraction of the AFL trophy case uncovered potentially-exploitable memory safety issues; by contrast, very few of the Rust issues were, with most being safe panics.
If by "stability" you mean "crash-proof-ness," I think there's no particular inherent reason Rust code is going to be more crash-prone than C, especially since basically the whole purpose of the language is increased stability. One notable shortcoming is that Rust doesn't currently have a fallible allocations API nor widespread support in common libraries for using it, so under memory pressure, if kmalloc fails, your only choice is to panic (Rust panic, i.e., unwind, maybe BUG() and kill the current thread). See https://github.com/fishinabarrel/linux-kernel-module-rust/is... for some discussion.
If by "stability" you mean interface stability, the Rust project has made great progress in the last year or two at stabilizing everything needed to write code that doesn't use the full standard library / link to a libc that can open files etc. See e.g. https://github.com/fishinabarrel/linux-kernel-module-rust/is... .
> 2) Security is often what Rust is expected to bring on the table. So is Rust actually more secure in the environment kernel requires? This could be tested by "clean-room" reimplementing something that is historically known to have many security issues.
Agree that in practice we're going to need to test this. But all of Rust's safety features (type system that handles null pointers, borrow checker, bounds-checked arrays, safe iterators so you don't need to bounds-check in the first place, etc.) work fine in kernelspace.
> 3) Are there showstoppers for kernel builds? Interoperability, build performance, architectures unsupported by Rust, etc. If so, could these be mitigated?
Build performance is a bit slow, but it's not as bad as userspace Rust because you inherently can't use that many crates and you generally don't want to be linking third-party code anyway - everything should be in the kernel tree.
For architecture support see https://github.com/fishinabarrel/linux-kernel-module-rust/is... . Notably, all the architectures that have kernels by the major distros (I checked RHEL, Fedora, Debian, Ubuntu, SUSE, Android, Oracle, and Arch) should work.
One challenge for interoperability is that most kernels in the real world are built with GCC, and rustc itself emits code using LLVM. The most common way of binding C code is using rust-bindgen, which uses libclang to parse C headers; even if you're not using bindgen, I believe you're still using LLVM's idea of C layout with #[repr(C)] structs and extern "C" functions. It's possible that kernels are built with particular GCC -m options that change the ABI (e.g., regparm) or GCC plugins (e.g., randstruct); if those aren't supported in compatible ways by LLVM / clang, then it's hard to write modules that load into an existing kernel. But, of course, if the question is to build new kernels with components in Rust, one workable restriction is to say that the C parts need to be built with Clang. There is good support for building the Linux kernel with Clang, and there are production Android models with Clang-built kernels.
> 4) How does it affect runtime performance? Average case. Bloat issues? Any pathologic cases? Any benefits?
There's a team that did some investigation on a prototype, and found that runtime performance was comparable, but binary size wasn't too great: https://mssun.me/assets/ares19securing.pdf (The prototype driver uses lots of unsafe code, but it's a good proof of concept for what ought to be achievable.)
I think we can claw back binary size with some focused work.
> 5) What other unexpected it brings on the plate? Both benefits and disadvantages. For example, could Rust types also be used to catch errors other than memory related, like invalid states?
One thing I'm very curious about is whether you can use a battle-tested third-party ASN.1 implementation (for example) instead of writing your own ASN.1 implementation in the kernel. (In fact the kernel has multiple ASN.1 implementations!)
Another useful thing is to use Rust's linear(ish) type system to prevent TOCTTOU bugs when checking userspace pointers, by making it very explicit when you're dereferencing the same address more than once.
A little closer to memory safety: Rust's Send and Sync traits make it easy to ensure you're not unsafely using data across threads (i.e., you're forced to pay attention to shared data and are unlikely to get into a big-kernel-lock situation), and you can use the typesystem to get a better and safer interface to things like RCU pointers. RCU requires that you be in a read-side critical section to read pointers; there's no concept of locking a specific pointer, being in one lets you read any RCU-protected pointer (as long as it's of the same RCU flavor, and confusing RCU flavors did lead to a use-after-free vulnerability recently!). In Rust it's pretty easy to have a Guard object on the stack that uses RAII to enter/exit the critical section, and ensure that you pass a reference to your Guard to any dereference of an RCU pointer type.
I guess that as more languages are added to a project, it gets harder and harder to understand and develop. You need to spend time learning yet another tool to be comfortable with inspecting the code and understanding the inner workings of it.
(3) can be a problem: Rust does not (yet) make it possible to define an ABI for Rust, which means that only by using external representations (C repr) can Rust code interoperate with non-Rust code. However, that's not a very big deal right now.
As to (1), ignoring toolchain stability (which affects C as well), the main concern would be (3) (see above).
Re (4), that's a legitimate question for sure. (My suspicion is that Rust will improve performance in general, mostly due to forcing better (public and internal) API designs on programmers. However, that's just hunch.)
Re (5), benefits. I think mostly it will bring this benefit: cleaner APIs. Obviously that won't apply to Linux's ABI to user-land, since that's not to be broken, but it could benefit the kernel in-tree. Re (5), disadvantages, I think mainly it's the learning curve.
Mighty nice of you to speak for all of us C programmers, so let me continue the story where you left off: The programer then tries it, gets annoyed with complaints about ownership in cases where the programer clearly did nothing wrong, and goes back to getting work done in C, putting the rust book into his long-overflowing "cute toys to play with later" pile
If you wish to use it, do. If you want to force your employees to do it, feel free. Telling everyone else how to live their lives and do their jobs is a dick move.
Great, just make a few unsafe definitions for your list implementation and the rest of your code can enjoy memory safety.
" So and So issue detected. If you believe this is not a bug, send this Snippet or A ST as a bug report (If you want, we can do this automatically for you!)"
And also "We've detected so and so issue. Please see our wiki <link to specific issue> for an explanation on what may have triggered this, and for ideas on how to rewrite such code"
https://hub.packtpub.com/rust-is-the-future-of-systems-progr...
Network chips has started to offer a common programming api as far as I understand Switch Abstraction Interface (SAI). Could one do this for more different than network hardware classes?
Unsurprisingly it's mostly C (95.6%) with a bit of C++ (2%) and Assembly (1.6%) and tiny bits of make (0.2%), shell scripts (0.3%), Python (0.1% and Perl (0.2%). I'd guess it'd be at least half a decade before Rust cracks 1% here.
That sounded strange to me, since the dislike of C++ within the Linux kernel is well-known, so I took a quick look. All uses I found were on user space tools, not in the kernel itself, and all of them (except for a "check if this compiles" test file) are in C++ because they have to call into libraries with a C++-only API (LLVM and Qt).
Linux kernel does not support c++ (its exceptions, RTTI etc). Only c++ I found was related to perf tool, 6 of 7 cpp files had test in their name.
I'm thinking machine opcodes in a .o file, to be linked wherever, are only of a concern if they're doing function call linkages (and maybe kernel space vs user space transition bookkeeping) improperly?
Not to mention race conditions in memory accesses.
Compile-time borrow-checking works for self-contained applications--the compiler has a complete picture of what's going on. Move that model inside kernel space--where blobs are being mutated across separately compiled modules, different chipsets (CPU/DMA/GPU), sometimes even in parallel--might as well wrap the whole thing in a big `unsafe` block.
Somebody's going to say "Oh, you're exaggerating. It's not that bad in Redox." Redox doesn't have to integrate with 28 years of kernel written in C.
I don't think there will be enough left outside of unsafe blocks to justify a second toolchain here. If the kernel were also in Rust, it would be a different story.
Rust side can add missing type information to C interfaces. Things that are "RTFM" for C, such as thread safety of the types involved and which function arguments are borrowed/owned, can be expressed on the Rust side even for C code, and automatically enforced when it's used via FFI. This adds a layer of safety to existing C code.
Adding loads of complexity (i.e. more opportunities to screw something up) for a minor gain.
And honestly, writing safe Rust wrappers around C APIs is not hard to do. The community has tons of experience with this. It does not wreck the coherence of the safe Rust code that uses those wrappers, like you seem to think.
You hear "X is good" from 7-8 people. You dismiss these people as fans lacking objectivity.
If you now hear "X has shortcomings" from 2-3 other people, do you now tell yourself "Yes I knew it all along. X isn't as good as the fans said it was" or do you think "Yeah, now that I have other opinions I can say that the majority of people think X is good and therefore it's probably good".
If it's the first, that's just confirmation bias. If it's the second, that's better but I don't get it - why does the mere presence of a contradicting opinion make the first opinion more valid?
In any case wouldn't you be better off trying X first hand and forming your own opinion?
To expand what I mean, when I recall what I have seen people say on Rust, some things strike me: overwhelming positivity, lack of depth, and it-cures-all-ills.
In a word, hype.
I don't owe anyone my attention to do some determination about how valid this or any other hype is.
If I hear people talking about something and the loudest and most numerous voices are hype, I think it is fair and rational to have a solid doubtful bias much moreso than if conversation seemed more varied, substantial, and rational.
Things that attract more rational people than hype-r people correlate with quality.
What makes people excited about Rust is not that it's perfect right now, nor do people claim that it's perfect. They're excited because they've seen a lot of progress in the last 3 years and see a clear path for it to become better in the next 3.
Almost every blog post I've read mentions shortcomings. Take one I read today from way back in 2017 (showing that people criticising Rust is not new) - https://onesignal.com/blog/rust-at-onesignal/. While they are very happy with their choice, they mention
1. Build times being very long. This is partly the fault of rustc generating verbose LLVM IR and partly because of code-gen macros like the ones used by serde (the crate that makes json parsing easy)
2. Libraries such as async clients for postgres and redis not being available. Http clients being immature and having to spend time improving them.
3. IDE experience being poor.
Of course, 2.5 years later many of these issues are no longer true.
* Build times have improved a 8-30% depending on the project in 2019 (https://blog.mozilla.org/nnethercote/2019/07/25/the-rust-com...). Incremental builds are a thing now, as are check builds (they generate no artifact, just make sure your code is compilable).
* Many more libraries available. Certainly all the basics will be covered once async-await becomes stable and all network libraries add support for it.
* RLS is reasonably good, IntelliJ-Rust is better and rust-analyzer is being worked on. Current situation is much better than the days of using Racer and will likely improve within 2 years when rust-analyzer is mature.
So there you go - an example of a Rust "fan" pointing out it's shortcomings 2.5 years ago, and the ways in which Rust has improved since then.
If your issue is that you see Rust being mentioned by fans a lot, it's likely because we keep reading stories about vulnerabilities that happen over and over, issues that wouldn't exist in a Rust code base. If these vulnerabilities were less common, maybe we wouldn't see Rust being mentioned so much.
There are bad projects with a lot of hype, bad projects with little hype, good project with little hype and good project with a big hype. Hype just isn't a good proxy for quality.
But hype is often a good proxy for success though: no matter how good your product/techno is, it's going to fail if there is no traction. Overall, if you think a product is good, it's good news if there is hype about it also. (And I think it's exactly what's happening with Rust: a good tool with a good amount of traction. But yeah, fanatics are always boring)
I think the biggest issues are reliance on cargo (the systems language that ignores your system libraries) and lack of a stable ABI, this is why it will never replace C. Many people find the borrow checker aggravating. Even rust fans seem to find the compile times far too long. As with all languages some will find it too high level and others too low level, should it throw exceptions or should it use error codes, etc.
And let's not forget the rust fans brigading any disagreement.
edit - do you mean in general or for this specific issue on the LKML, my reply assumed in general.
Rust doesn't have recoverable exceptions. It has fancy error codes. It is also quite low level in the same spirit as C++: only pay for what you use, and what you use should not be possible to write more efficiently by hand.
Playing devil's advocate: when compiled with panic=unwind, it has. However, the fact that when compiled with panic=abort they turn into unrecoverable exceptions tends to make the "should it throw exceptions or should it use error codes" choice clear.
In Rust, you'd use Result where in C++ you would use exceptions.
Panic is equivalent to assert and should not be used where you'd use exceptions in other languages.
It's still pretty new, but it's gaining adoption quicker than what I would have expected. It's now used in Google, Facebook, Amazon, Dropbox and Microsoft. Most of the time it's for some niche or experimental project, but it's still an interesting trend.
> I think the biggest issues are reliance on cargo (the systems language that ignores your system libraries)
This is not accurate. There is no "reliance" on cargo, you can just use plain rustc and it would work. And you can of course link dynamically to a system's library: by default Rust's binaries are even linked with the system's libc actually, and AFAIK most external C dependencies are linked that way (bindings to openSSL for instance).
> and lack of a stable ABI
Many people would love to have a stable ABI, but it's way too early for that because it would freeze Rust development, while there are still a lot of things that need to be improved in the language.
I agree with you on compile times though and I hope things continue to improve in that regard.
Basically that's what I was trying to say but failed somehow.
FWIW I had been hearing buzz about Rust from the community that I work with regularly (including the occasional "I'm working on rewriting $COMPONENT in Rust") long before I started frequenting HN.
A common pattern to mitigate this (though not always applicable) is a generic trampoline to a monomorphic function for the cases where the genericity is mostly a matter of convenience e.g. let's say that your function works on strings, but for convenience you take a `T: AsRef<str>` (callers can pass anything from which a string reference can be created cheaply), the first thing you do is `let s = s.as_ref()` so the vast majority of the function is monomorphic (it takes only one set of concrete types as input). Rather than this:
fn my_function(s: T) where T: AsRef<str> {
let s = s.as_ref();
// do stuff with s: &str
}
fn my_function(s: T) where T: AsRef<str> {
_my_function(s.as_ref());
}
fn _my_function(s: &str) {
// do stuff with s
}
That's because the primary (and so far only) use for Rust is as an excuse to avoid learning C++.
Nobody actually writes real software in it.
As you doubtless already know, the programming language everybody likes is a programming language nobody uses.
Personally, I find it obnoxious when people start new projects in C code because they're comfortable with the language and unwilling to learn something better, and those projects contribute to the toxic cesspool of vulnerable software that is undermining civilization.
I can understand what you mean, though. It's really lame to hear "that's how we've always done it". My personal opinion, however, is that language's like Rust are still taking the wrong approach because we should be making the computers manage the memory for us. Much of the guarantees from Rust's type system could also be gained by more advanced garbage collection or designing hardware in conjunction with garbage collectors to get around some of the thornier issues in that field.
If you could take out the perceived drawbacks of garbage collection, then there isn't a reason to do manual memory management. That's my personal soapbox, though, and I could go on.
Cheers.
(Swift basically does what you're talking about, for example)
I agree that for some applications the overheads of GC are fine and wrangling lifetimes is tricky enough you should use a GC-based language, not Rust (and probably not Swift either, since it's easy to write Swift-style reference counting in Rust).
However --- Rust lifetimes are about more than just memory safety and performance. They also enable data race freedom and strong control over aliasing, which prevent other kinds of bugs and can enable higher degrees of optimization. They enable affine types, which let you write APIs with very nice static guarantees, e.g. that a Nonce value is only used once. I don't think we're fully exploiting Rust lifetimes yet.
It's also important to remember that for many applications Rust lifetimes are hardly any burden at all. For lots of code I write --- serial, synchronous code manipulating off-the-shelf data structures where the overall shape of the heap is tree-ish, or DAG-ish and reference counting is acceptable --- I hardly have to think about them at all.
Indeed, how dare they spend their own free time doing things you find obnoxious? The scoundrels!
> and unwilling to learn something better
They also seem to not be willing to blindly accept your judgement instead of their own in terms of how best to implement things they wish to! How inconsiderate of them!
That's not the obnoxious part. The obnoxious part is where their vulnerable software gets deployed and subverted to harm to third parties.
> They also seem to not be willing to blindly accept your judgement
I certainly wouldn't ask anyone to blindly accept anyone else's judgement. The evidence is in.
Speaking the way you do, though, you just sound like an ingrate. Take a moment to thank all those people writing that software you use, instead of telling them that they are being obnoxious.
When I buy something from a shop and their database gets hacked via some bug and my data is extracted, I'm hurt even though I had minimal control over the situation.
Vulnerable software is a negative externality, like air pollution.
2. When it comes to your own choices, information about what language is used by every facet of a product/service isn't usually available.
3. Even when information is available, most people don't and won't choose stuff based on language anyway, which means if you do care, you can't expect a viable alternative to be produced by the market.
So the standard libertarian advice is inapplicable in a triply redundant manner.
I see that there is one use-after-free bug, and lots of out of range access bugs. How does that happen?
> I see that there is one use-after-free bug, and lots of out of range access bugs. How does that happen?
The use-after-free bug was in explicitly unsafe Rust code (via the "unsafe" keyword). I assume that code was intended to be a performance optimization, but I haven't looked at the details.
An "out of range" bug here is typically an array-index-out-of-bounds, which results in a panic (safe abort) at run-time. Panics aren't great but they're not exploitable.
The use after free [1] is because of unsafe code, specifically, a function that says "trust me, I know what the type of this is."
I only checked a few of the "out of bounds" bugs, and they all are panics [2], which are not memory unsafe. Of course, someone could cause a real one with the use of unsafe.
1: https://github.com/shepmaster/sxd-document/issues/47#issueco...
This will not check your dependencies. You can use other tools, like cargo-geiger https://github.com/anderejd/cargo-geiger to check your dependencies.
And of course, at the lowest levels, doing anything useful requires unsafe, since your operating system doesn't expose a Rust API to do tasks.
Likewise we are also aware that 68% of Linux kernel exploits in 2018 were caused by memory corruption bugs easily preventable in any systems language with bounds checking enabled by default.
And, sorry, we don't all know that it isn't a magic bullet. Thus the people who seem personally invested in this stuff like its a conspiracy and the one true thing is being kept down by the man.
Again, I've seen your other comments. You know full well that it's a problem that other systems tackled with language and hardware support. Why is Rust the one true way when there are other approaches that have worked in the past?
Did you see that other comment about using C as "a toxic cesspool that's undermining civilization"? That's a tad much, right? It's a type of technical myopia. Software isn't undermining civilization. We are, by having perverse incentives in multiple areas of life that make it more profitable to prioritize one thing over another.
It's also the same kind of stupid that makes people think they are a freedom fighter because they use Debian.
It's also boring. So, yeah. Thanks.
I've seen very little C/C++ code where I can't find tons of bugs. I've been in the business for over 25 years and gotten tired of this a long time ago. Enough is enough. Even the smartest people in business don't seem to be able to use C/C++ safely. (Yes, C++11 etc. did improve matters, but not enough).
Personally I'm happy there's finally at least one language that is effectively runtimeless, C/C++ like performance, significantly more secure than C/C++, C ABI compatible and has reasonable momentum. Rust might be it.
It doesn't need to be a silver bullet for everything, just give me more memory and concurrency safety.
If it's Rust, fine. If it's something else realistic, fine. It's not cure all, but borrow checker is clearly a major improvement for low level safety.
I know that old C++ got some bad reputation in the past but I think it's time to reconsider that with the changes that started with C++11. The only area where Rust really is better is the management of lifetimes and its associated higher memory safety.
I think the point is that it would be worse because it's closer to C.
C++ has quite a few powerful features that Rust doesn't have, including better platform support, but Rust has a lot of things going for it that would make it well suited to kernel development:
* no exceptions, and language features and a type system that is set up for explicit error handling - the kernel would need to disable C++ exceptions anyway, making error handling just as cumbersome as in C.
* the ML inspired type system helps to write correct code (eg sum types)
* the language is not riddled with UB and legacy C support induced cruft. You can opt in to unsafety with `unsafe` blocks, but these can be tightly scoped to where it's necessary and can be especially scrutinized
Especially in the context of a kernel, the increased safety guarantees are worth a lot.
We're still pretty early in Rust's lifecycle, and there is still a lot to be learned about what `unsafe` can really break, especially at-a-distance. The UCG WG is doing this work in [1], but I can understand if the kernel developers want to hold off on using Rust for more central parts of the kernel until this work is farther ahead.
I think it might be at least half a decade before Rust starts being used in more central parts of the kernel (if at all) because adding a second language significantly complicates the build process. Also, it is blocked on Rust supporting all the platforms that Linux supports just as well. It would be disastrous for Linux to drop support for it's long tail of platforms because Linux could no longer be built for those platforms.
I don't have links at hand, but there were already instances where a bug in an `unsafe` block had effects in completely different (and seemingly random) places. Discovering all the ways in which `unsafe` blocks can cause unsafe or undefined behavior in unrelated places is still an active field of research.
> It would be disastrous for Linux to drop support for it's long tail of platforms because Linux could no longer be built for those platforms.
Which is why driver modules are a good place to start. Drivers are specific to certain pieces of hardware which are oftentimes only used with CPUs of a specific ISA.
unsafe blocks can cause UB period, UB means the program is broken but the UB can manifest anywhere.
C or C++ don't make this any better, they just make the entire program into a source of UB.
They're just places where you're telling the compiler "I know what I'm doing", once an unsafe blocks has created an UB thing can break anywhere.
It is though.
> let's suppose that your unsafe code has a presupposition that cause an UB if not met. If this is a bug
It is, or the code should not present as being safe.
> but is-it always possible without performance issue? If not, then you have to audit all the usage of the unsafe part.
If it's not possible to fix it (or if you don't want to fix it) then the wrapper for that unsafe code should also be unsafe, and it should document its assumptions such that callers can know what to look for.
The tautological contract is that safe rust is safe. If it's possible to trigger UB by passing the "wrong" value to a rust function then that function is not safe and must be marked as unsafe itself. An unsafe block means the compiler trusts that you know what you're doing, which is different from lying to the compiler, which is what you're apparently advocating / defending.
Only code in unsafe blocks can cause unsafety (ignoring already and yet to be discovered soundness bugs in the compiler [1]). But the effect can easily materialize in any location that uses the unsafe code, or types that go through it.
To me this is somewhat obvious, but it's true that this is easily overlooked and should be communicated.
To express this better in the type system, Rust would need an effect system.
[1] https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Ais...
I believe the kernel core developers are good programmers and good at code reviews. That said, a huge proportion of Linux CVEs are memory-safety problems -- use-after-free, race conditions, out-of-bounds access, etc -- which do not exist in safe Rust.
> I can understand if the kernel developers want to hold off on using Rust for more central parts of the kernel until this work is farther ahead.
I can understand this too! It takes time for large communities to change, and the only real research we have on `unsafe` is the RustBelt paper, which demonstrates that the concepts of the borrow checker are sound provided that `unsafe` code respects its invariants. The way this framework has been pitched, though, is for building optional modules. If everyone takes this seriously, I think it'll result in wins all around -- Linux benefits from memory-safety improvements, Rust benefits from kernel developers' experience, and the world benefits from having more secure code running in ring-0. I'm looking forward to this.
Rust's borrow-checker, obviously, prevents that problem. Certainly it's possible to write C++ code which is as efficient -- and in some cases more efficient -- but will it actually happen?
For example, the Rust compiler can and does reorder fields in structs to improve packing. In C and C++ you have do it manually, and for C++ templated types you sometimes can't pick an order that's optimal for all type parameters. (Rust can pick different field orders for different monomorphized types.)
The Rust compiler does other nice representation optimizations, e.g. Option<bool> is represented as a single byte with three possible values. Not just a hack for Options, but general.
Maybe even more importantly, Rust is really strict about aliasing and this can improve optimization. E.g. a variable that's a mutable reference to T ("&mut T") cannot alias any other reference to T in scope. An immutable reference to T ("&T") can alias other immutable references to T, but the data is truly immutable (unlike a C or C++ const reference). This completely subsumes "type-based alias analysis" and also C/C++ "restrict", and is stronger than both. This information is potentially really useful for optimizers, but unfortunately, since LLVM is mainly for C/C++, the Rust compiler can't take full advantage of this aliasing information yet :-(.
Yikes! That can be disabled, right?
> The code generation part ends up being nice when something goes wrong. When somebody sends in an oops, I often end up having to look at the disassembly (and no, a fancy debugger wouldn't help - I'm talking about the disassembly of the "code" portion of the oops itself, and matching it up with the source tree - the oops doesn't come with the whole binary), and then having code generation match the source makes things a _lot_ easier.
That said as long as you don't build graphs with reference counted types it's hard to leak memory by accident.