Rust for Linux redux
lwn.net
lwn.net
That's an odd statement. What CVEs aren't due to programming mistakes? I'm not sure if the majority of CVEs for the Linux kernel come down to memory safety, though I would not be surprised, but certainly a huge number are.
> without a clear and obvious benefit beyond promises that can only truly be fully fulfilled with a whole kernel written in Rust.
That's not true, really. You don't need a completely safe kernel to have an improvement to safety. If every device driver were memory safe tomorrow we'd be better off.
That said, I think this will be an interesting, possibly losing, battle. The Linux Kernel is extremely monolithic, it has a lack of testing and code review, decades of dug-in investments, and a strong history of not prioritizing security or even considering it to be a legitimate goal. Fixing that seems like it will itself take decades, whereas the current approach really feels like it's trying to get it done ASAP.
If they can do it, cool. As a Linux user I'll possibly benefit. I'm curious to see how it plays out.
Is that really true? I mean my first reaction is that the linked article / thread is a direct counterpoint. All the review and testing the rust facilities are going through.
Are these changes not going through substantively the same process as other proposed changes to the kernel?
And my reviewing of a few drivers source short commits is enough to tell me that those delegates do not perform a satisfyingly thorough review by any mean.
Heck, I saw patches of just a couple dozen lines which exhibited bad copy-pasting errors anyone without prior knowledge could have spotted. You don't need to know what the code does to spot some, you don't either need to know what the driven device does to spot some: purely formal errors with bad macros definitions for example. This kind of stuff wouldn't even pass the first internal review where I worked, which just looked at formal appearance (then there were more in depth reviews, and then there was an external review, but we'd make as sure as possible that our code would be clean before going out).
So first you have people (employees of company A) which sends code to a public, external project without having done a proper internal review. Then you have someone else (employee from company B) who claims to have reviewed the commit but hasn't done it properly or at all. And then possibly a third someone who validates this, but doesn't actually check either. It has become a job, a task like another, with the same people who do the same sloppy job as quickly as possible to get rid of it and go home earlier or slack, in the same proportion that you can find in any other position in the world.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
Look at the seq_file thing Qualys discovered the other day. The overflow was obvious if you thought about it, and all Qualys did was think about it. But the bug was present since 2014.
https://www.qualys.com/2021/07/20/cve-2021-33909/sequoia-loc...
Linus's law is empirically untrue for security bugs - many eyes don't actually spot them. Moreover, we have computers, which are good at doing repetitive and detail-oriented tasks with 100% accuracy. Why not use them?
> a strong history of not prioritizing security or even considering it
is that really true ?
Some years back there was a viral blow-up where Linus basically said, "security is important but there are lots of things that are important." A lot of people in the security field decided that meant "not even considered" even though that's ridiculous. Linus has always had a pragmatic, holistic approach to the kernel, and many specialists hate that because they think their field is the most important and all others should be second.
If security wasn't even considered, would Linux really have become the de-facto base on which high security orgs (banks, 3 letter gov agencies, etc) deploy? I doubt it very much.
As a security engineer who has seen egregious security malpractice on the part of developers, I fully agree that there can be a real problem with that. However I think it's silly to suggest that the Linux kernel has a history of not even considering security.
Linus had multiple statements that won him pwnies, but what I'm referring to is decades of mailing list posts where he's insulted researchers, or decades of him and Greg rejecting CVEs and hiding vulnerabilities, etc. This has persisted even today, mostly from Greg, but in a less public way than it once was due to cultural shifts.
Make no mistake, Linus and Greg have always had a hostile relationship with security researchers.
> many specialists hate that because they think their field is the most important and all others should be second.
Another straw man. I never said anything like this; that security should be the number one priority or that anything else shouldn't be a priority.
> would Linux really have become the de-facto base on which high security orgs (banks, 3 letter gov agencies, etc) deploy? I doubt it very much.
Is this satire? Are we really going with "Banks deploy Linux... therefor it's secure" ? Did you know banks also run Windows XP on their ATMs?
Linux has had external contributions to security, yes. Much of that has been despite upstream, and with immense work across decades to get upstream to play ball.
> However I think it's silly to suggest that the Linux kernel has a history of not even considering security.
Sorry but the only way for this to be the case is to simply not know the history of the Linux kernel.
> That's an odd statement. What CVEs aren't due to programming mistakes?
I guess they meant as opposed to higher level design mistakes.
Rust's safety promises don't prevent logic bug CVEs in the kernel. But I think it would be a major improvement if the kernel were written in mostly safe Rust.
If the demonstrated complacency of Rust coders toward potential bugs is factored in, and inexperience with the language among potential reviewers, use of Rust in the kernel could actually increase the number of exploitable faults. I do not assert this would certainly occur, but no one can demonstrate it would not.
https://www.chromium.org/Home/chromium-security/memory-safet...
https://langui.sh/2019/07/23/apple-memory-safety/
these companies apply things like sandboxing and fuzzing to reduce the incidence of memory unsafety bugs, and yet they're finding a majority of their security bugs being memory unsafety. if you can't find memory unsafety in your c++ code, it's because your code isn't worth attacking.
The overwhelming prevalence and impact of memory safety bugs has retared development of solutions for the remaining, nontrivial problems for many decades, by consuming so much of the attention.
lol what?
This is so very the opposite of my 6+ years of experience in Rust, where the community has a serious focus on testing, and great tooling support for it as well.
I honestly can't take anyone seriously who thinks that adding Rust to the kernel will increase security bugs. I just can't imagine they know anything about Rust, security, or the kernel.
I do not doubt that there are Rust coders who are especially vigilant about introducing bugs, but they certainly are, as in all times and places, the exception.
This isn't a real thing either. I see it asserted all the time on HN and it's hilarious.
> Anyone hinting that bugs are still possible in Rust gets downvoted to oblivion (as here).
90% of the time it's because the person is making a stupid point. 10% of the time it's the community being annoying.
So just to reiterate, you're basing this on being barely an observer on forums, and I'm basing this on multiple years of professional rust development.
Shellshock would be a good candidate: Bash is designed to be able to pass around some amount of shell scripting in environment variables, which obviously leads to some pretty severe security issues if attackers can control environment variables (say, CGI scripts). So you can argue that the problem here is a design mistake rather than a programming mistake.
?
Did you mean the Windows kernel?
Linux has had incredible focus on security. SELinux being a prime example.
> To me, security is important. But it's no less important than everything else that is also important!
> - Linus
> I think the OpenBSD crowd is a bunch of...
Apart from that, if there is no test coverage it is difficult to talk about security or reliability (I don't know myself if there are tests and how good they are, I'm just assuming GGGP is right).
He acknowledged that and did so, years ago.
He himself wouldn't disagree with that.
In an OS where Linux kernel is actually an implementation detail in what concerns userspace.
SELinux is just an LSM built decades ago, I wouldn't say that somehow proves that upstream cares about security.
Spectre
No, as far as I know, the design mistakes which lead to Spectre (and other similar vulnerabilities) are not on the microcode; these design mistakes are on an even lower level, in the hardware structures which execute both simple instructions (which are decoded directly, without going through the microcode engine) and microcode instructions. Most of what the microcode "fixes" for Spectre and similar do, is flipping a few "chicken bits" (to disable or bypass some of the hardware structures), and providing extra semantics to a few of the complex instructions (which go through the microcode engine) like LFENCE and VERW; these changes do not actually fix the problem (which is on physical hardware), but instead give software ways to workaround the issue.
You should argue instead that the programming in question is the VHDL or Verilog (or other proprietary language) which was used to generate the hardware.
This feels like putting the cart before the horse. "we shouldn't integrate Rust into the kernel in a modular sort of way cause it's not as optimal as more thorough type-aware integration."
Like, cross that bridge when you come to it, eh? The kernel is currently comprised of hundreds if not thousands of components, talking to each other via C API/ABIs. And talking to hardware is almost entirely ABI. That is how the kernel do.
Would we benefit from more type knowledge across boundaries? Absolutely, but this is hard (especially given how different the c and rust approaches to allocation are), and shouldn't stand in the way of progress.
One gotcha, though: a stronger type system encourages the removal of runtime checks. When doing a partial conversion, I've found it's best to leave any existing runtime checks in place, even if the type system makes them "redundant", because it doesn't really until the calling code has also been converted. Once it has, you can go back and remove them.
The best situations for using Rust modules from C are simple APIs that allow the complexity and danger to be strongly encapsulated.
- It is not fun but frustrating to work in Rust, and contrary to C, you are limited by the language/compiler on what you can do.
- building/compiling the kernel is not trivial, and you will add a new huge dependency that you have to deal with to build the kernel for whatever target. Let's suppose you want to build for MIPS, then you need to have Rust supporting MIPS.
As an exemple, there is a common package in python that decided to start having their module in Rust instead of C. Now a lot of users of the module are pissed off, with good reason, because you can't build/install the module anymore in a little bit older or non conventional distributions.
I'm following a couple of projects that transitioned to rust, and my experience as a contributor is not stellar. A minimal rust project can take hundreds of mb of disk space just in dependencies, and double that for build space. The solution for some has been providing build bots, but again doesn't help me as a contributor, where I need to be able to rebuild from source.
This has on me almost the same effect produced by large projects: I only contribute if I have a large vested interest in the package, otherwise I just avoid because it's time consuming.
Nothing stopping you from building rust code with manual vcs, rustc, autoconf, and make, just like C.
Would it be emitting objects that gcc/ld would link against?
Well, this is a working patch set, so, yes, though I have no direct involvement and haven't read all of it yet.
> I imagine it would have to use llvm-rustc,
Yes, it uses upstream rustc.
> is gccrs ready for the job?
Not yet. They're making great progress, but major chunks of the language are still missing. They'll get there.
Using upstream rustc isn't a blocker for new code implementing drivers, but it is a blocker for getting into older drivers, or the kernel. The blocker is platform support; or at least, the current largest blocker, and either rustc will gain it, or gccrs will be good enough to compile this codebase. We'll see :)
> Would it be emitting objects that gcc/ld would link against?
Yep, it emits output that looks like any C compiler's output, you link 'em together and you're good.
If you manage to compile the kernel with clang, in theory you can even get cross-language LTO; this is working in Firefox, but I'm not sure if anyone's tried it with the kernel yet or not.
Note, that there is still bunch of unsolved issues [1] in LLVM to allow all building all of Linux kernel. The efforts had stopped for some years but recently gained steam again, though a lot of time was wasted before.
https://security.googleblog.com/2021/05/integrating-rust-int...
https://security.googleblog.com/2021/06/rustc-interop-in-and...
You can pass -v to Cargo and it will show you the rustc invocation it makes if you want to reverse-engineer what Cargo does.
As a noob, I had to wade through endless "but don't do that, just get the latest from Cargo!!!!" when I asked for advice on how to use my system-provided Rust packages for my project.
Moreover, in the Python world a distinction is made between "software that runs your system" and "software that you use for development"; maybe Rust people think similarly.
Sure. That's inherent with software.
> Moreover, in the Python world a distinction is made between "software that runs your system" and "software that you use for development"; maybe Rust people think similarly.
What?
Other folks are masochists and get their jollies from cryptic errors, segfault debugging, pouring through valgrind traces, and of course manually managing memory. If you aren't suffering, are you really programming?
I tease, but not entirely. Sometimes I do in fact enjoy the challenge of C programming. But it's squarely type 2 fun.
Those languages give you syntax and/or runtime tools to get away with badly structured programmings (allocating / deallocating stuff like mad, lots of implicit behaviour). C is not like that. It wants you to think and learn how to structure programs (this is transferable knowledge, i.e. it gets much better with time).
You can translate most C to Rust automatically (https://c2rust.com/) and there's nothing that I'm aware of that can't be done in Rust via unsafe and transmute. (technically some things like specific label jumps can't be translated, but all of those can be rewritten to other constructs) Do you have some specific cases in mind?
But yeah bottom line, nowhere does Rustc "stop" you from doing things. Just strongly discourage :)
Writing correct C is very hard and most definitely not fun. It's like juggling with with 7 balls and if you drop one you'll be shot. C is defined for a weirdo abstract machine that doesn't match what computers really do, and when people apply their intuition and knowledge about computers to their C programs "because C is low-level", it's a crapshoot whether they will trigger undefined behavior and the compiler goes off the rails with wild optimizations.
If I designed a low-level language I would enable such optimizations by making it easy to communicate your precise intent to the compiler. Not by making the standard a minefield of undefined behaviors.
I'm pretty sure I mention this in the LWN comments, but, since it gets repeated so often the contradiction might as well be repeated as well:
No. Unsafe Rust only gets to do three things that aren't related to the "unsafe" keyword itself. It can dereference raw pointers, it can access C-style unions, it can mutate global static variables.
That's everything. Your C program is free to define x as an array with four elements and then access x[6] anyway - but Rust deliberately cannot do that. Not in Safe Rust, but also not in Unsafe Rust either. Writing "unsafe" doesn't mean "Do this anyway" it only unlocks those three specific things I mentioned, and so sure enough x[6] is still not allowed because that's a buffer overflow.
In fact by default the Rust compiler would warn you, if you write unsafe { foo[z] = 0; } that unsafe isn't doing anything useful here and you should remove it. That array dereference either is or, if z is small enough, is not, an overflow, and either way unsafe makes no difference.
For instance, if you want to overflow a buffer intentionally,
fn main() {
let mut a = [1, 2, 3, 4];
let b = [5, 6, 7, 8];
let ptr: *mut i32 = &mut a[0];
unsafe { ptr.add(5).write(999); }
println!("{:?}", b);
}
[5, 999, 7, 8]
(Note that this is not just extremely platform-specific and compiler-specific about whether a is in front of b or vice-versa, it is straight-up Undefined Behavior because you write past the end of an object... but the equivalent C code is also Undefined Behavior, and subject to the same LLVM optimizations. So if you were happy with the corresponding C code, this is the equivalent Rust.)
If you really, really want, you can write your own UnsafeSlice type that does the unsafe stuff internally and exposes the standard indexing operator, which would make foo[z] actually accept arbitrary indices just like in C. But you shouldn't. https://play.rust-lang.org/?version=stable&mode=debug&editio...
(Among other things, a code reviewer should be suspicious of your use of "unsafe" in the internals of a thing without stating why the higher-level abstraction is safe, and in fact the abstraction is wildly unsafe here, so it's bad style to write code that launders the unsafety, so to speak. In the Rust for Linux patches, there are "SAFETY" comments above each use of "unsafe" defending their logical safety.)
https://play.rust-lang.org/?version=stable&mode=debug&editio...
UnsafeSlice is a terrible idea, but let us at least give it the normal ergonomics of a wrapper type so we can say UnsafeSlice(&mut a) rather than needing curly braces to make one :)
> Your C program is free to define x as an array with four elements and then access x[6] anyway
It's free to do anything, but you can't be sure that it will do that, because of the utterly weak specification.
Woot?
fn main() {
let a = [0, 1, 2];
let _b = [42, 42, 42];
println!("{}", unsafe{a.get_unchecked(6)});
}
That was C code. The rust code for that was provided by drran.
> The point is that `[]` is always bounds-checked, and the bounds-checking cannot be opted out of even with `unsafe`.
I don't think anyone is hung up on whether you use std::ops::Index or not. You can access arrays without bounds checks.
>No. Unsafe Rust only gets to do three things that aren't related to the "unsafe" keyword itself.
>[...]
>Your C program is free to define x as an array with four elements and then access x[6] anyway - but Rust deliberately cannot do that. Not in Safe Rust, but also not in Unsafe Rust either.
>[...]
>In fact by default the Rust compiler would warn you, if you write unsafe { foo[z] = 0; } that unsafe isn't doing anything useful here and you should remove it. That array dereference either is or, if z is small enough, is not, an overflow, and either way unsafe makes no difference.
By explicitly doing it so, in an operation that is easy to grep for, or in the case of a binary library, search for the symbol during the linking phase.
Something that is impossible to validate in C, unless one is using a custom compiler, like Apple is doing for iBoot firmware.
Most of the time, these operations will be inlined, so they will already be gone by the time it gets to the linker. The compiler phase is the latest point where they are still visible.
what's next, you can't write 6[x] in rust but it's perfectly fine C, thus rust is inadequate?
Hint: I am not tialaramex, and neither of us said anything against Rust in our comments.
Dereferencing a raw pointer. One of the three specific things I said unsafe Rust can in fact do.
This is not a "funny syntax" thing, a[6] is the idiomatic and obvious way to express this in Rust, and, it isn't allowed because it's a buffer overflow. Whereas a[6] is also the idiomatic and obvious way to express this in C and the result is Undefined Behaviour.
Not what you want in a kernel.
Deeply subjective. Rust has been the most loved language on Stack Overflow for 5 years in a row now.
> add a new huge dependency
Sure, but setting up Rust is much much easier than GCC with all the trimmings.
> As an exemple, there is a common package in python that decided to start having their module in Rust instead of C. Now a lot of users of the module are pissed off, with good reason, because you can't build/install the module anymore in a little bit older or non conventional distributions.
Assuming you are referring to pyca, you are mistaken and there has been a lot of misinformation about the change. Rust support is needed to build the module, but not install it. Pyca works just fine for users without rust and works everywhere rust does. Users on niche CPU architectures which haven't been sold commercially for 15+ years were the only ones impacted.
How many of those votes were cast by programmers who have actually used the language?
https://insights.stackoverflow.com/survey/2020#technology-mo...
Maybe the Python number is smaller because those people are working right? All the Rust programmers are just working on a fun program at home, and Python is a serious language now, you're working the 9-5 with Joe Coder?
Haskell is 51.7% though. So, I guess when I wasn't looking Haskell really exploded in boring office environments and I shouldn't expect to see any more hobbyist Haskell projects everywhere... or your hypothesis was just wrong.
Maybe it's just that Rust is new and exciting, let's look at what people want to use that they don't now, surely that'll be Rust too and we'll know it's just Hype.
Huh, that chart is dominated by Python. 30% of programmers not doing Python wanted to start.
> why does Rust report 86.1% love while Python only got 66.7% ?
I mean, because python is in practice a boring office language where the immense majority of devs have to maintain Joe Coder's 2009 Django set of custom attributes.
> Huh, that chart is dominated by Python. 30% of programmers not doing Python wanted to start.
Because people believe that if they learn python they'll instead land a cool ML job which pays 100k more than what they have ? Like, I have my girlfriend who literally does not know anything about programming ask me if I could teach her python because she saw an ad about it (for a "land a CS job in 3 months" type of thin). That necessarily causes some inertia for Python, not enough to be more hyped than rust, but enough to influence results.
Think about the 2016 movie "Deadpool". Deadpool is a one joke character. There are no major Marvel characters in the movie, the stakes are low, there is no connection to the larger Marvel Cinematic Universe storyline. Reynolds has played this exact character before, in a movie which nobody liked. There was inevitable fan hype before it came out, "the merc with a mouth" sells comics and those fans are going to see the movie whatever, but fans don't know anything right? But, both critics and audiences seem to have liked this movie, a lot more than its studio expected. It made a lot of money and got pretty great reviews from most quarters. Neither of those things is hype, that's called success.
The Loved result looks exactly like what I see what I talk to people about Rust. Those who haven't heard of it of course aren't looking to write Rust, for those who've only heard of it, it's on their "things to check out" list with Go, and maybe Swift but it doesn't jump out at them. But among those who've written Rust you see a spike of enthusiasm, "Hey this is really good".
When I learned Go, I filed the acquired skills away. "This may be useful in some future scenario, but I have meanwhile ceased to be employed writing TLS bit-banging code for which I thought Go would be the best option, so, never mind now"
But when I learned Rust the first thought was "I should write more Rust". I immediately rewrote the smallest interesting C project I ever published and pushed that to GitHub. Then I wrote a bunch of code to check some of my intuitions about Rust's safety when used by people who are unreasonable (misfortunate, a collection of perverse implementations of safe Rust traits). Then I started writing a triplestore, which I'd done twice before in C -- a friend and colleague left programming and went into management after his third one, so cross fingers that doesn't happen to me.
This isn't true for the overwhelming majority of deployments, since pyca/cryptography was/is distributed as a pre-built wheel. There is no runtime dependency on Rust in pyca/cryptography; the only downstream change is that packagers are required to have a Rust toolchain.
For now. One day I'll wake up and a future version of this container will refuse to build because whatever library pulled pyca/cryptography in got upgraded and now needs the Rust-only version.
You probably know this but for people reading along who think using requirements.txt is the same thing: it is not.
How lockfiles work is that you define your dependencies in a file like pyptoject.toml or Pipfile (similar to a Cargo.toml). You then use pipenv or poetry or pants to compute all the dependent versions of your dependencies and transient dependencies. Then that's saved in a lockfile. Any time you need to remake a venv for local Dev or rebuild a docker container or install deps for CI is uses the same locked versions from the lockfile. Only when you decide to recompute the dependencies do the transient dependencies change in the lockfile.
Sadly, a standard lockfile was rejected from PEP-650, held back by pip being woeful:
https://www.python.org/dev/peps/pep-0650/#a-standardized-loc...
> Additionally, pip would not be able to guarantee recreating the same environment (install the exact same dependencies) as it is outside the scope of its functionality.
Well then, maybe fix it? Because clearly it’s an issue? A good chunk of that explanation really reads like “ehhhh, can’t really be bothered fixing this”, which makes sense given the Python devs approach to the last couple of Python versions: no fixes for anything important, just more half-baked features nobody asked for.
Oh god, tell me about it! 'Hey guise I heard pattern matching in rust and Scala and Haskell is popular! Let's add it to python but with no compile time checks to make sure matches are exaustive!'
Some excellent and smart devs who I really do respect worked really hard to deliver a complete dog shit feature while pip languishes for almost a year with a broken version resolver [1]. It's so frustrating. :( :( :(
This is kind of the root of the problem, when you are kind of a hobbyist dev wanted to work with the latest shinny new version of everything, then everything is fine.
But when you try to do different things that are not mainstream or doing embedded, then you understand why sometimes you have to keep old software or hardware.
Rust like go are made for a connected world, where you are always upstream, always connected to get the latest versions, and it's ok to do breaking change to the language every few years.
Did you ever try to build a Linux from scratch? Trying to fit some build or runtime constraints? Then you learn the real cost of each dependency that is added and of their complexity.
Imagine you had that much dependencies to build a kernel, probably this much of memory, CPU and storage used. Now you need to add rust, its dependencies, other tools related to it. Each dependency possibly not supporting your system or configuration or require their own library dependencies in a new version that is maybe conflicting with older versions already used by the system. And that you can't change without breaking other existing programs in the system. And maybe you can fix these other applications to support the new version of the library but you just wanted to "update" your kernel and not spend 10 days reworking your system unexpectedly because a stupid update required it.
But that being said, most of the time, issues are not coming from really exotic "chips" but from little variations, configuration or system library versions.
Let's say you want to cross compile from x to y, and want code to use that specific memory space. This is when things start to get messy usually.
Example of how you can lose a lot of time and get crazy, when you just wanted to compile the code of something for your case:
https://stackoverflow.com/questions/67902309/how-to-compile-...
https://stackoverflow.com/questions/31492799/cross-compile-a...
If you wrote a driver for HW that only appears on 1 or 2 CPU architectures then targeting is less of an issue. I would not be surprised if lots of drivers in Linux only work on x86 anyway.
[1] https://github.com/antoyo/rustc_codegen_gcc
[2] https://github.com/rust-lang/rust/pull/87260
Can you give it a try?
(Note that MIPS is a Tier 2 support for Rust, which is a commitment to keep it building but does not obligate running tests for every checkin).
In my personal experience (since I wanted to see how big of a problem this is), I looked into bringing up Rust as a cross-compiler for Mac OS 9. This requires a compiler that can emit PowerPC machine code, as well as a toolchain that can handle XCOFF objects and classic Mac OS's strange resource formats (if you ever wondered why Win32 has resources, that's why). Retro68k provides such a toolchain (albeit GCC based), and I wrote a rustc target file to make it spit out XCOFF objects in PowerPC format.
Then I got hit with a bunch of llvm assertions about unimplemented functionality in it's XCOFF generator and gave up.
Less anecdotally, the ArcaOS people (responsible for trying to keep IBM's freakshow fork of Windows and DOS alive) and TenFourFox both have abandoned attempts to maintain Firefox forks for OS/2 and old Mac OS X (respectively), specifically because of the Quantum update making Rust a requirement to build Firefox.
I heard Rust did merge in a GCC backend, which might help some of these retrocomputing projects... but there are platforms out there where the primary (or only) development environment is a proprietary C compiler. (e.g. Classzilla uses Metroworks to provide old Mozilla on Mac OS 9) I'm starting to wonder if some kind of "Rust to C" backend might be useful for these cases...
Linux also can't abandon hardware support for some of these weird environments, either. So until and unless Rust-with-GCC can compile on every environment Linux does, we aren't going to see anything more than Rust drivers.
Can't because why though? I agree it shouldn't abandon them just to get more Rust, but there are other reasons some of the crustier less used platforms go away.
But why not? Are we obligated to support everything forever? If it’s hardly being used, and is starting to get in the way of safety and correctness improvements, why can’t we drop something old, arcane and unused?
But it is used by retrocomputing enthusiasts. Linux has been supported by them for the platforms they care. They gladly hold up the bar “now you have to support Linux yourself”, with C compilers already existing and being supported by someone else. With Rust becoming a build-time dependency, things suddenly turn into “you're not getting any Linux until you port rustc to your platform and then make sure it's working there at all times”.
When you get into embedded, you will start to see all sorts of weird arcane setups that actual businesses rely upon. Case in point: this commercial kitchen appliance that is actually a DOS PC built with modern parts. [0]
Not to mention the startlingly high number of large businesses running off of IBM server hardware. Much of that is actually legacy stuff that's been rebranded and massively upgraded over the years. A company that bought into System/360 in the 80s or AS/400 in the early 90s will almost certainly have backwards-compatible zSeries or System i hardware running literally 30-40 year old programs.
Point is, there's lots of business critical crap running on things other than x86 or ARM. I only used retrocomputing as an example because I had a good anecdote for it. Businesses treat computers as if retrocomputing was also somehow mission critical and they pay handsomely for the privilege.
But like, isn’t that a them problem? If you want to calcify your compute layer, don’t be surprised when the rest of the industry moves on, and possibly does things that aren’t compatible anymore? If they want to keep running that software, I think it’s their responsibility to either evolve their software to keep up, or deal with the fact that they’ll have to run their own old version/fork when the time comes.
If they contribute to the kernel, I would have thought their perspective would have been represented on one of these threads by now, as they seem to get posted at pretty much every event hahaha. Maybe they do and I totally missed it as well.
People who have had prior exposure to C tend to find Rust frustrating.
People who have not had prior exposure to C tend to find it fun, and an average programmer of this sort can fearlessly write bare-metal code that beats the code of the best C programmers writing in C in safety and rivals it in performance.
Systems programming has been fundamentally changed by Rust. It's become as accessible and democratized as web programming, no longer the sole province of a cadre of elite C programmers.
Servo was not meant to ship (at least by Mozilla, when there was paid staff working on Servo), it was a research vessel and Rust components now shipping in Firefox (Stylo, WebRender) started life in Servo.
That depends on the beholder.
Developers that grew up in Algol derived languages like Modula, Pascal and Ada, feel Rust brings fresh wind into systems programming, with safety considerations we thought it were lost forever and only partially covered by C++.
Then there are the others like Kernighan, that feel that languages like Pascal, sorry Rust, is programming with a straightjacket and better not change anything.
My other criticism is the bad statistics used. It is like I create the "robot athlete" and I compare it stats with the average of all athletes including the young children and the people with some physical problem. You should compare self driving with cars with exact same safety features and save driver demographics. Bonus if you calculate all deaths caused by illegal driving and then ask the giants WTF not put the money into first solve the speeding and drunk / tired driving , I bet ANN work better on this problem.
Rust in Linux seems to me a waste. IMO the Unix philosophy is great but it needs a better implementation , one that is based on the present day hardware and expectations.
Why would they want to do that though? No one would buy a car with those features. I guess you could get a few people to buy one if the insurance was way lower, but you certainly wouldn't get decent market penetration.
- a law that requires it in all cars
- a law that requires it in new cars
- a law that requires it only for new drivers (less then 2 year) and for people caught speeding/drunk.
- a law that will make certain roads only available for self driving cars or for people with this safety system. It would be a compromise between AI drivers camp and people that still want to drive themselves.
Maybe you will say something about privacy, this system can be implemented with no connectivity to spy on you. It could be read only when you want to pay your insurance or in case of an accident.
But also AI can be implemented in a different way, like you could have some quality cameras recording traffic and have the AI detect who is using the phone, who is not paying attention , who is moving in a erratic pattern , so it would be an advanced spending camera.
The argument is that the AI driver camp will demand humans to be removed from the road, because their AI driver is better then the average. To prevent you losing the right to drive then this average needs to be improved and this can be done by removing the bad drivers and a good place to start is the ones that do not respect the rules. Otherwise you might object to install a safety system in your car but then the big companies will lobby hard and you will have to use the Tesla/Google or Apple AI powered cars, then not only that he government will know all about your movement but the ad companies will know it too.
The same effect applies every time a new language is introduced, if it doesn't completely replace the previously used language. In this case, Rust won't.
Zig might be a better fit, given how much more similar to C it is.
It'd make your argument a lot stronger if you can actually list some of these negative consequences. "Another languages to the stack" and "Different than C" is just complaining about change because it is change.
What negatives have already been shown that isn't just about "This isn't C"?
But here, you're already starting with 2: C and assembly. Besides inline assembly, a small but very important part of the Linux kernel is written in assembly on every architecture: the system call entry point (entry.S) and the kernel entry point (head.S). And if you consider each architecture's assembly as a separate language, it's more like 10 languages than 2 languages. I'm always impressed whenever I see changes to for instance signal handling or thread flags which touch not only the common code in C, but also the entry point assembly code for each one of the many architectures Linux supports; whoever does these changes need to not only know the assembly language for all these architectures, but also have at hand all the corresponding tooling and emulators to compile and test the changes.
The story here is pretty different: integrate a new, high-level language into a 30 year old, 30mil SLOC, production code base, that billions of people rely on every day, AND actually extract some value from that work.
It's not as if C is going to disappear from the kernel as it's something like 25 million lines of C code, and if Rust was to be supported, the current C experts who are maintaining various subsystems will now also have to become Rust experts, so that they can effectively accept or reject code contributions in that language.
Personally it just seems illogical, better to make a new kernel in Rust is you really want to use that language, than converting small parts of a HUGE C kernel. Google has been pushing for the inclusion of Rust into the kernel, it's weird that they are not writing their own shiny new Fuchsia OS kernel in Rust, instead of C++.
[1] https://twitter.com/cpugoogle/status/1397265884251525122
[2] Like a two years project.
Rust isn't terribly hard to learn, especially for a kernel developer with a good understanding of C and of memory. You can pick up the basics in probably an hour. A lot of its design choices match approaches the kernel already takes (traits are like ops structs, errors are reported via return values, etc.)
And Rust is a language that plenty of college students pick up for fun. Professional kernel engineers should be able to learn it just fine. Frankly the hardest thing about Rust is that it makes you think deeply about memory allocation, concurrent access across threads, resource lifetimes, etc. - but these are all things you have to think deeply about anyway to write correct kernel code. If you have a good model for these things in C, you can write the corresponding Rust quickly.
In fact, learning Rust and thinking about Rust's concurrency rules has made RCU a lot easier for me to understand. RCU is famously a difficult concept, but the kernel uses it extensively and expects people to use it. So "requires little knowledge and is easy to understand" is not an existing design goal of the kernel - but having people pick up Rust might help there anyway.
(Zig seems like an entirely reasonable choice too. Send in some patches! :) )
I'm not so sure. I program in C for a living (embedded, for almost 20 years) and believe me that I tried learning Rust, but when I see something like:
_dev: Pin<Box<Registration<Ref<Semaphore>>>>
Source: https://github.com/Rust-for-Linux/linux/blob/rust/samples/ru...
Until it fixes use-after-free, better keep using C anyway.
But Zig also improves on C's safety in many ways, not least checked arithmetic enabled by default in safe release modes, along with bounds checked memory accesses, function return values that cannot be ignored, checked syscall error handling, explicit allocations, comptime over macros, a short learning curve and remarkable readability.
It's hard for systems programmers not to appreciate any of these qualities in isolation.
But hey, its spirit lives on most managed languages, Julia, Closure, WebAssembly text files and plenty of other stuff Lisp based.
Rust is known to be hard to learn (YMMV), but C is even harder. If things go according to plan, someday for some use-cases you'll be able to contribute kernel code in pure safe Rust without having to learn C. In the meantime, adding Rust doesn't seem to be such a big ask when you consider what the kernel already has beside C: Assembler, the "C preprocessor (yes, it's actually a different language independent from C, and some kernel macros are really complicated), the BPF an io_uring APIs (essentially their own DSL), and a myriad of other inner-platform curiosities you might need to deal with depending on the kind of kernel work you do.
Concerning Zig, the cons may be smaller then Rust, but so are the pros. IMHO it's not worth it in the current context (I like Zig but it seems "too little, too late" to me). But there's no telling until somebody puts in the work for a "$OTHER_LANGUAGE in the kernel" RFC like is currently happening for Rust.
There's bad to be weighed against the good. Adding complexity strains and breaks processes, slowing development. Among other things, this means additional bugs and lets them survive longer, so it's not even clear rust is a win purely from a bug perspective.
That, but in the opposite direction.
Decades of promises of self-driving cars. And still nothing able to drive without a driver.
There have been small improvements...cars that have autonomous abilities in some cases.
But overhauling the entire driving fleet of the world to use 5-year-old technology....it's not a realistic expectation.
There are smaller, more practical expectations.
Waymo have driven 20 million miles autonomously since 2009, and as of late 2020 claim 74,000 of those were done completely driverless. They're still a far cry from being common, but they're here and they're impressively safe.
Sounds familiar
There's a perception issue here: people think that Rust people (not necessarily the maintainers or official evangelists, but the community at-large) think Rust is as close to perfect as you can get because of its safety features, because these people talk about Rust like it's a universal problem solver.
Despite the plus sides, there are lots of incumbents, certain domains are better served by managed languages, and regardless of our wishes C and C++ have 50 years of history.
Even if we stop writing new code in those languages today, Cobol shows us how long we would still need to keep them running anyway.
Microsoft, for example, despite their newly found love with Rust, is full of job ads for C++ developers in green field applications.
In my experience, the community of actual Rust users is much more level-headed. While most do love the language and the "this aspect of Rust is irrefutably better than the equivalent in $OTHERLANG" opinion occasionally pops up, the community seems pragmatic and well aware of Rust's cons. Case in point: the "should I use Rust" questions on the rust subreddit don't get dogmatic answers, and often result in "Rust isn't ideal for your use-case" advice.
The standard economic analysis of cooperatives is that, being run by their employees, they find it hard to embrace disruptive change (e.g. change which involves firing people). Hence, you probably buy your food from a shareholder-owned business, not an employee-owned cooperative.
Linux is a bit like a cooperative in that decisions are made by the software engineers, not by shareholders or managers. In particular, most Linux contributors are probably heavily invested in C. If Rust gets the big boost of being allowed into the kernel, that could make C - and their own skills - be perceived as less valuable, maybe even obsolete eventually.
I'll do some economic imperialism here, and claim that Rust's technical merits or otherwise are a second-order issue. Linux developers have an interest in keeping C dominant, so that will probably happen.
That kind of argument might hold for the average junior javascript dev, but kernel developers are normally an experienced bunch and C is rarely learnt today as a sole language.
This is in some sense a co-operative, but it's first and foremost technical. A co-operative farm feels a duty to consider Jim's irrational dislike of corn when deciding what the farm should grow. On the one hand, this land is very suitable for corn, there is a need for it, and they have all the equipment, on the other hand, Jim hates corn. But the technical focus means that although Jim's dislike of corn is respected in the LKML, it doesn't override the technical decision that clearly we should grow corn.
Among non-experts you can end up with lengthy arguments about how "good" C programmers don't make the mistakes Rust defends against. Those arguments won't last five minutes on the LKML because everybody there considers themselves a "good C programmer" and has made those mistakes.
One big technical obstacle to Rust was allocation. In C all your allocation is explicit, so that's kernel policy. A line of code that seems to simply assign a to b, for example, must not in fact secretly allocate resources. The ordinary day-to-day Rust you write does have some implicit allocation, but the Rust-for-Linux people landed changes so that you can have core and much of alloc in Linux but lose features that have implicit allocation.
Unless there's a hitherto unforeseen technical blocker, Rust in Linux seems inevitable at this point. They have a list of unstable features they're requiring -- expect over the next months either things get crossed off (having meanwhile become stable) or removed from Rust-for-Linux in favour of an alternative. Unlike Firefox I'm guessing Linus has no appetite for living on the bleeding edge, so I think once Rust-for-Linux is just stable Rust, it'll make its way towards the Linus tree.
Regardless of what Linux kernel team decides to do.
I see Rust having more success in OSes, and platforms, that aren't so reliant on being UNIX clones.
I'm increasingly convinced that a good chunk of MacOS/iOS security problems are due to the simple fact that it is very difficult to write correct system level code w/ C.
If (and it is a big if) the performance of rust can be in the same neighborhood as C - and these kinks can be worked out, then as a architect, I think it might be time for a ideological come to Jesus moment/protest reformation of the kernel.
> void crypto_aead_clear_flags(struct crypto_aead *tfm, u32 flags)
These are valid due to poor type usage, but make no sense:
crypto_aead_clear_flags(..., PAGE_SIZE)
crypto_aead_clear_flags(..., EBUSY)
crypto_aead_clear_flags(..., SOME_RANDOM_THING)
And for info, Android Treble drivers are mostly C++, although Java is also an option.
Ah and Fuchsia is a mix of C++ and Rust at it lower layers.
Although I still haven't really played around with Rust yet myself (still trying to grok the new C++20 features in recent weeks), the fact it puts your code into a (sometimes admittedly annoying) sandbox to prevent these bugs could be a godsend.
Mixing in the new flavor into a codebase that is 95% (?) using (and abusing) C with warts and all does not strike me as a good idea.
Interaction between assembly, C and Rust is not nearly well, enough understood to start.
Now if it does go ahead, it will be a great platform to figure out the edge cases, bugs and all those things nobody thought about.
I am in favor of Linux.NEXT
Write a new kernel in Rust (if it's the best choice)
Take all the learnings from Linux and all the research in operating systems that have occurred since and make Linux.NEXT.
Steal from Linux, WindowsNT, Quebes, OS/400, Solaris, Minix, z/OS Plan 9, Fuchsia, BeOS/Haiku, ESXi, QNX, and of course TempleOS
Creating an operating system suitable for 2020 with the way technology, infrastructure, connectiveness, pervasiveness, privacy conscious, built for the hostile world of the Internet.
Some of those in the list are using C++, and now adopting Rust as well, actually.
I do not advocate "stealing code" but stealing ideas and inspiration :)
The perfect OS doesnt not exist today and it never will.
But we can do better than we are today.
But maybe we could start by actually catching up with having full stack OSes written in safe systems programming languages.