Rust heads into the kernel?
lwn.net
lwn.net
(This specific post:)
https://news.ycombinator.com/item?id=26812047
https://news.ycombinator.com/item?id=26831841
(Previous Rust-in-kernel work:)
https://news.ycombinator.com/item?id=24334731
Linus Torvalds on Rust support in kernel - https://news.ycombinator.com/item?id=26831841 - April 2021 (290 comments)
An RFC that adds support for Rust to the Linux kernel - https://news.ycombinator.com/item?id=26812047 - April 2021 (261 comments)
Supporting Linux kernel development in Rust - https://news.ycombinator.com/item?id=24334731 - Aug 2020 (354 comments)
Linux kernel in-tree Rust support - https://news.ycombinator.com/item?id=23800201 - July 2020 (491 comments)
Linux kernel drivers in Rust might become an option in the future - https://news.ycombinator.com/item?id=20833639 - Aug 2019 (254 comments)
(I'm thinking of making HN's software render links to previous threads in this format automatically. If anyone can think of a downside to that, please let me know - maybe at hn@ycombinator.com so as not to take this thread off topic.)
Right now both alloc/std and even basic language constructs like indexing into a slice can panic. With little help from the tooling to guard against it. This makes sense when considering that Rust evolved as a language for writing somewhat higher level applications like browsers.
While Rust and especially the library ecosystem is generally great at forcing you to handle application domain error conditions, that mostly doesn't extend to these lower-level concerns.
But I'm hopeful that this can improve the entire ecosystem over the medium term. With things like lints that forbid panics, more focus on custom allocators and fallible allocations in std and the library ecosystem, etc.
edit: just to clarify on the slice index panicking: my point isn't that the C situation is better, but that a big reason for moving to Rust would be to benefit of a safer, more convenient language that doesn't require such extreme carefulness as C.
So the language should help in preventing panics, or making it obvious where they can occur. This is perfectly possible in Rust by using accessors that return `Option<T>` or iterators. But the tooling for such use cases can be improved, for example with lints, and the API surface can be customized to mostly prevent such conditions with the type system.
AFAIK they're not using std. And isn't this kind of panic the kind of thing that causes BUG()s today? Probably can just map one to the other.
Panicing on invalid slice indexing is a non-issue.
Well, it's less of a issue that what we already have. It would still be nice to be able to tell the compiler: "Issue a diagnostic for any [some qualifiers] panic that isn't optimized out by unreachable code analysis.".
Actually most of the time you probably wouldn't want a Vec that reallocates automatically in the kernel, you'd probably want a dynamically allocated array of fixed length the majority of the time.
This isn't the question here. malloc (or other memory operation) failures should never panic. According to Torvalds[1]:
> Allocation failures in a driver or non-core code - and that is by definition all of any new Rust code - can never EVER validly cause panics. Same goes for "oh, some case I didn't test used 128-bit integers or floating point".
Kernel code is a bit quirky in this sense that it will always favor system stability (and expect proper errors to trickle down) instead of crashing. Also, Linux runs on plenty of low-memory embedded systems so failing on a malloc may not even be a particularly rare occurrence.
"Rust is obviously a better choice for Linux than C." "Ok, if it is obvious, it should be easy to demonstrate?" This is not only about Rust - 99% or so of all new tech can't demonstrate any benefits.
If you want empiricism try academic papers instead of mailing lists ;) Some numbers: most critical bugs in the Linux kernel are due to memory safety: 40 out of 65 bugs allowing for code execution found in Linux in 2017 could have been prevented by using a memory-safe language [0]. The paper is about Go, but the same logic applies for Rust. Fun fact: 39 out of these 40 bugs were in drivers [1], so I think starting with drivers in Rust is a good idea.
[0] https://www.usenix.org/conference/osdi18/presentation/cutler (really good read!)
[1] https://arxiv.org/pdf/1909.06344.pdf (disclaimer: I'm a co-author of that paper)
If anything, software development is sandcastle-building. We all have ideas on how to build it, but at the end of the day we're just amateurs fucking around with plastic shovels.
This. And even then, after the calculations are done the engineer add a “safety margin” (and it's not 10%, more like 500 or 1000%).
It was Microsoft that got people used to software failing on a routine basis, and "have you rebooted?". Before, anything that ever crashed, you sent back for a refund, or expected a tech to come out and fix it post-haste. Clearly MS could not make money that way, but getting people with no experience to believe failure was normal was quite cheap to arrange.
Apart from that, I think you are giving too much credit to construction engineering outside of the highly regulated sectors. They mess up just as often, and expensively so. But stakeholders in construction are slighly more aware that their requirements are going to be set in stone, pun fully intended.
What really makes a difference is the malleability of software. This is an invitation for feature creep and last-minute changes. Also, the impact of overengineered, unmaintainable software is not quite in-ya-face as for physical assets. Therefore, in software engineerin project managers need to be religious about these matters. And software engineers must get better at communicating about these concerns.
To address the particular point being made here, lots of things in EE are experience driven, for example which decoupling capacitors to use or to not use 90 degree angels on pcbs. I think many aspects of typical engineering are way more driven by experience and vague rules of thumb than we expect looking in from the outside.
The reason why this is more obvious in software development may be because it's pretty hard to have good measurements and understanding failures is pretty hard and working around them is pretty easy.
Claiming that an entire science field "lacks empiricism" is quite brave, and definitely false in the case of CS.
Perhaps you are talking about software engineering in practice instead, but that is false too (for any serious project).
> "Nevertheless, we believe that, even today, the advantages of using Rust [in the Linux kernel] outweighs the cost." Where is the evidence?
Three sections of the RFC are spent on characterizing the sentence you quote. The LWN article is not the RFC.
Furthermore, many of the claims I wrote in the RFC are simple facts (e.g. features that Rust has) that you can easily corroborate. For other claims, you can look up plenty of articles, scholarly and otherwise, about Rust benefits.
But none of that really matters, because the only way to gather the evidence you are actually requesting (Rust in the Linux kernel) is getting into the kernel and evaluating Rust after a while.
> "Rust is obviously a better choice for Linux than C."
The RFC does not claim it is "obviously" a better choice.
Quite the opposite, in fact: see the "Why not?" section.
> Perhaps you are talking about software engineering in practice instead, but that is false too (for any serious project).
I'm not the first one lamenting the lack of experimentation in CS. See https://www.cs.princeton.edu/~jrex/teaching/spring2005/fft/m..., http://www.inf.fu-berlin.de/inst/ag-se/teaching/S-Komponente..., https://ieeexplore.ieee.org/abstract/document/1027796, https://ieeexplore.ieee.org/abstract/document/4221632
> Three sections of the RFC are spent on characterizing the sentence you quote. The LWN article is not the RFC.
I know, but it doesn't contain (empirical) evidence.
> But none of that really matters, because the only way to gather the evidence you are actually requesting (Rust in the Linux kernel) is getting into the kernel and evaluating Rust after a while.
That's the medical equivalent of giving the patient an unknown drug to see what happens. Did the patient get better? Maybe it was the drug! Did the patient get worse? Maybe it was not the drug!
A better way could be to fork a less-visible (but still well-maintained) C project and rewrite it in Rust. If the defect rate over time is significantly lower than for the upstream C project then that is empirical evidence in support of a Linux rewrite in Rust. Running such an experiment would take a lot of time and effort, but that is no excuse for not doing it.
Take your first IEEE paper's abstract:
"Empirical software engineering research needs research guidelines to improve the research and reporting processes."
They are pointing out problems with the empirical research that is being done, not claiming "almost complete lack of empiricism in Computer Science" as you did.
There are issues in all science fields, their research, their statistics and their reporting, in particular outside their major journals (and not even those are infallible). Calling out an entire science field in particular is quite insulting to all their researchers.
> I know, but it doesn't contain (empirical) evidence.
Empirical evidence of what? Please do not remove/ignore parts of my reply to suit your argument.
You may claim that you would prefer the RFC to include references to papers and articles; but you cannot claim the evidence does not exist ("99% or so of all new tech can't demonstrate any benefits") or that an entire field has no clue ("the almost complete lack of empiricism in Computer Science").
> That's the medical equivalent of giving the patient an unknown drug to see what happens. Did the patient get better? Maybe it was the drug! Did the patient get worse? Maybe it was not the drug!
No, it is not equivalent, because we are not unconditionally adding Rust into the kernel (much less "rewriting it" as you claim). In fact, one is free to have a C and a Rust version of the same driver and compare whatever metrics you need. Even the RFC contains such an example and I am aware of at least one company doing precisely that.
Moreover, Rust is not an "unknown language" in the sense of an "unknown drug". On the contrary: it was explictly designed to address some issues such as memory-safety.
> A better way could be to fork a less-visible (but still well-maintained) C project and rewrite it in Rust. If the defect rate over time is significantly lower than for the upstream C project then that is empirical evidence in support of a Linux rewrite in Rust.
Not really, because 1. you would need to show the same results would apply to the Linux kernel and 2. we are not rewriting the kernel in Rust.
> Running such an experiment would take a lot of time and effort, but that is no excuse for not doing it.
As mentioned, I am aware of at least one company running an even better experiment than the one you propose.
If you want faster results, then you will need to put money into it yourself.
When someone thinks their hashmap is faster in theory they get asked to benchmark it... it often isn't in practice.
But when someone decides that some norm in whitespace, use of parenthesis, or use favored control-flow concepts will result in a lower defect rate or reduced maintenance costs we somehow are expected to accept these conclusions without clear evidence.
Even though everywhere is in computer science where we can easily measure results there are countless examples of theoretically better stuff that works less well in practice. Why should we expect our intuition on human interactions to be any better than our intuitions over which data structures are faster?
But even in medicine, probably the area with the most extensive empiricism (we have insane amounts of data on mouses and humans), there are claims that won’t ever get to the point of doing a study on. There are stupid things that get tested, eg homeopathy (it has plenty of studies showing it is quack), but I think noone would say it is irresponsible for a doctor to claim that hand healing over a gun shot is not effective, even without direct evidence.
Of course there are surprising findings everywhere and as other commenters wrote, especially in low level optimizations the “obviously slower” solution can easily win. But I would wager that C is so terrible as a language with so little protection from even trivial mistypings, no real abstraction power requiring text-based macros that simply should have never existed to begin with, and the worst of all, not even goddamn namespaces making everything’s name indecipherable. On the other hand, I’m not necessarily bought that Rust’s borrow checker is the best way for handling memory-related errors, for example a syntactically better-than-C language like C++, or Zig might reduce memory bugs just as much, we don’t know.
I do hope that Rust doesn't suffer from an explosion in complexity and compile times, though. Memory safety and overengineering shouldn't have to go hand in hand.
And many of these were _bad_, C++ was a different language back then and the culture of how you write C++ was different. There were times when it was fashionable to write OO spaghetti almost as bad as the stereotypes about java, or templates that worked except for some unforeseen footgun (then you could template libraries that worked correctly but figuring out how was a trip). And before all of that there was a decade when C++ compilers were just _full_ of bugs, and before that there was no standard and every compiler was different. The negative stereotypes about C++ come from all that.
I think he had every reason to fear that e.g. he would have to fight someone who implemented control flow using an inheritance hierarchy because it's a pattern. And he could find himself in the minority.
(Also I'd say there are still counterproductive fads, and it's still common to write C++ that's beautiful on the outside and completely inscrutable inside, or generally pay a huge price in code size and complexity for marginal gains elsewhere, or to depend on optimizations that don't trigger. Same in many other higher-level languages but somehow not in C.)
Second, more important, i don't remember any talk of adapting C++ to bare metal programming, while there is an ongoing cooperation with rust folks already.
C++ already is used in bare metal environments and in at least one production kernel (XNU) so there just isn't any adapting to do. The standard explicitly distinguishes between hosted and freestanding environments and defines what you should expect to have available.
That does not mean you would try to use virtual functions (much), or std::vector or std::shared_ptr (ever). Instead, you would use abstractions tailored precisely to the kernel environment, and arrange that wrong code would tend to not compile, so that when code compiles, it works. In C, you have to enforce all conventions by obeying instructions in comment blocks, and updating all uses every time the comment block changes. In C++, you can encode the rules directly into the type system and put that to work, not just checking correctness, but actually generating correctness.
This is the same as one would do in Rust modules. Failing to make full use of Rust's type system would be a grave mistake that we need not fear will happen.
Since I'm on mobile, here's a link to a previous explanation of mine: https://news.ycombinator.com/item?id=1422160 It is meant to be read in conjunction with the original post, addressing what I saw as the point most people were missing, not a complete restatement of his points.
Your post also mentions operator overloading, which is a bit nastier since the calls aren’t explicit. But Rust also has operator overloading, so similar complaints would probably apply? In practice I’d expect the kernel to be fairly strict about limiting operator overloading (which would apply for both C++ hypothetically as well as Rust).
I can't directly speak to Rust's overloading support. If it's still one static type lookup he may be ok with it, plus you have the clarity around ownership of any intervening constructions. I'm sure the Rust Evangelism Strike Force (awesome 80s cartoon music intensifies) will be along presently to answer the question of how much assembler one can generate with an overloaded operator.
C++ really does have a particular ability to stack a huge number of abstractions into one little line. Ultimately it's a continuum rather than a sharp line, but C++ is really far on one side.
I did get your point but would like to point out that using these slurs isn't helping any discussion.
There are fanatics in every area everywhere. Their existence doesn't devalue any of the bigger groups they belong to. Their existence is a fact of life.
I've worked with plenty of Rust devs (and I am an aspiring one myself although not all the way there yet) and I've only ever seen hard-working people with an excellent eye for detail and desire to maximally utilize the hardware and have better safety than other languages they worked with in the past. Zero fanatics.
Just at the moment, C++ is notably better at this, in important cases, but I gather there are plans for Rust to adopt more of C++'s capabilities. The Rust designers fully understand the cases I mention, but are very busy.
Linus's ignorant diatribe about C++ tells us plenty about Linus, but nothing meaningful about C++ or its suitability for kernel coding. That every detail he rails about in C++ applies equally to Rust is unintentionally revealing. That each such detail makes C++ and Rust better for large systems, moreso.
Take some vintage C++ code from late nineties, like games, and there's no concept of ownership, before smart pointers and move, and full of global's everywhere.
It was also when code were transitioning for a multi-threading era, so using the same code style of the previous sequential era was really bad.
Of course, all the new features C++ gained since '98 only make the language more useful for coding large, long-lived systems, as is amply demonstrated in the tens or hundreds of billions of lines of C++ code implementing exactly those systems. Rust's adoption of so many of those same features increases its usefulness for a growing number of those same purposes.
To me C++ is a language of last resort. It can do everything you need but you should not touch it if there is any other option.
Building from your metaphor, I would add:
Python et al.: normal household batteries; everyone can use/operate them, not much power but can be used with little to no training and a screwup happening means you went looking for trouble
Java et al.: electric batteries; substantially more power but has to stop relatively often (GC pauses), everyone can use the things running on them at least reasonably well [0] but one little thing can potentially cause a catastrophic chain reaction (one cell out of thousands => thermal runaway (eg. the infamous NullPointerException))
Rust, Go et al.: industrial backup generators or hydroelectric plants; substantially more power yet again but have a startup/setup time (learning a more complicated language/ecosystem), as reliable as can be realistically done but have to be spec'd right for the building/installation/etc. at hand (time vs. resources tradeoffs); screwups happening means you probably went looking for trouble (eg. in Rust: excessive cloning of data, building of circular references without weak references; not so sure about specific examples in Go but didn't want to lump it together with Java et al.)
C, C++ et al.: "nuclear power [...] very powerful but dangerous if only used slightly wrong" like in your metaphor; should only really be used by experts but even they are not safe from introducing catastrophic failures with something seemingly harmless (seen as such at the time of writing the code); it gets safer over time but there is the huge inertia of existing installations running with huge time and resource costs associated with dismantling or even modernisations; footguns are everywhere and not always immediately recognisable especially since there recommendations about usage and safety concerns floating around which are sometimes years or decades out of date
[0]: AFAICT a substantial part of the business software world runs on Java code cobbled together by not-necessarily-full-on-experts
PS: My personal experiences/biases:
- learned C++11 for a while, didn't like it, SEGFAULTs and associated debugging/fine-tooth-combing over the some-tens-or-hundreds-of-lines long code got really annoying really quickly
- learned Java, Haskell and Prolog in Uni; Haskell was a nice intro to functional programming but nothing I would want to use even occasionally, Prolog was meh, Java was/is a convoluted, resource-hungry mess with huge amounts of boilerplate for every little thing and will IMHO always be associated with bad assignments in class(es)
- Python3 I use every other week or so but not for "real programming (TM)", but instead for small (automation/download/crawler) scripts; I like it but for serious programming in a business environment - ie. something I personally would want to rely on for money - the duck-typing is just too unstable for my liking, even though I still like it for rapid prototyping of concepts coming into my head which I then later may or may not transfer into:
- Rust... currently and personally my favorite language for "real programming (TM)" even though I struggled with the borrow checker for a long, long time (and still run into it on occasion); sometimes really long compile times but nice runtime performance in most cases even without special care and tuning while I still have the confidence that if it compiles it runs and any errors encountered at testing-/runtime are probably more due to logic errors than running into something like "oops, in this deep branch of cascading ifs I accidently didn't set some object's field or dict's value meaning at some later point the program blew up due to NoneError or something similar"
- learned a bit of Go (before even finding out about Rust) but I just didn't like it, it just didn't "gel with me", don't even really remember the specific reason(s)
Also,
> but has to stop relatively often (GC pauses)
is severely outdated and I don’t see how NPEs are something specific to Java.
Shorter pause times than Go.
These are bad comparisons.
Google chose C++ 17 as the implementation language of the Zircon kernel for the fuchsia OS.
https://fuchsia.dev/fuchsia-src/development/languages/c-cpp/...
PS: C++ compile times are actually shorter than rust compile times.
For this kind of project its more important to have people with pratical experience in kernels, than experience in a particular language.
These people will have a experience in languages they used before, so even if the decision was taken today, you wouldn't find enough people with experience in kernels AND Rust, but will have much more people with experience in kernels C and/or C++, so C++ would still be a good default in comparison to C.
It would be pretty amateurish to get the kernel expert folks to learn Rust while implementing it, as it would probably mean that the end result in Rust would be worse than the end result in C++ as they were more comfortable and experienced in that language.
It will take more time to form specialists in some key areas so that decision can be made with less side-effects.
https://news.ycombinator.com/item?id=26831841 (292 comments)
https://news.ycombinator.com/item?id=26812047 (262 comments)
It is worth looking at all the serious bugs in deficiencies in the upcoming 0.8 milestone: https://github.com/ziglang/zig/milestone/10
This is not a criticism of Zig, nor is it an insinuation that Zig is unable to write correct software. It is a reflection of that fact that compilers are very complex.
Any argument that Rust, by contrast, would be OK in the kernel applies at least as strongly to C++, because C++ can read kernel C headers, where Rust will need hand-maintained equivalents. (Some automation is possible, but not enough.) The FUD Linus flings about C++ applies equally to Rust. (Anybody who says Rust is less complicated does not know Rust well at all.)
But argument validity is not what carries the day: bad arguments tend to fail, but good arguments get pot luck. Ultimately, it comes down to biases of those with power to decide: C seems to them like a familiar quantity (full of risks, but favored risks), while Rust happens to be hot just now.
In three years' time, Rust could very well not be so hot, and a new new thing might take over its social role. Zig could conceivably be that thing. (Zig isn't "safe", by Rust's definition, but by then something else may be more important; anyway the lack of safety does not disqualify C.) The new hotness will necessarily already exist today, to be mature enough to consider then, but not necessarily one that immediately comes to mind. Being different from Rust could be, by then, among the new hotness's attractive qualities, much as percieved differences from C++ boost Rust in some milieus.
Zig is totally different from Rust. For starters, it does not have a safe subset of the language that prevents an entire class of bugs. It's slightly nicer C. There are plenty of languages like that.
> [Zig is a] slightly nicer C. There are plenty of languages like that.
Are there “plenty of [actively supported] languages like that?” The design space of relatively simple, typesafe, imperative native programs strikes me as rather unexplored since the early 90s. At least I can’t think of many languages I would consider a nicer alternative to C - indeed actual C alternatives like Pascal and Fortran are arguably less nice. Maybe this is my ignorance.
There are various languages “between” C and C++ in terms of abstraction and complexity, and the space of “somewhat C-like” languages is heavily explored (D, Java, C#, Objective-C, and in a different sense Ada or Swift). There are plenty of alternatives to C++. But Zig is one of the only ones that’s not dramatically more complex than ISO C.
Happy to be corrected if I am missing something obvious (I mostly do functional programming).
Also, the more familiar one is with a subject, the less one tends to conflate it with "plenty" of others. Zig is a completely different language to C (while also targeting the same systems space where you would need "unsafe" Rust to get anything done). In fact, it's hard to find too many languages that come close to Zig's comptime, plus the entire array of safety features offered by Zig. Your argument is a weak strawman at best.
Zig might have some safety features, but it's dwarfed by what Rust has to offer: https://scattered-thoughts.net/writing/how-safe-is-zig/
More importantly, you can cause exploitable crashes in Zig programs while in Rust you need unsafe for it (at some point in the program).
And yes, each Rust codebase has unsafe somewhere, but there are quite many codebases that don't have a single line and instead use dependencies that encapsulate the unsafe in a safe to use API.
Would you say also that Rust's macros offer a stronger type system than Zig's comptime?
There is this misconception that Rust will make systems secure simply "because memory safety". But this overlooks the most foundational point of systems safety: simplicity. Most failures or exploits (two sides of the same coin really) originate through excessive dimensionality, a point where Rust's complexity unfortunately dwarfs Zig's.
The kernel project distributes several userspace utilities and new ones of those might be worth writing in Zig.
However, each has their place. One man passion projects should not be used in anything mission critical. They totally should be used for fun, enjoyment and inspiration, and they may potentially grow to industry scale.
Zig is newer than Rust, but the Zig Software Foundation was in fact founded before the Rust Foundation, and already employs several core members. For example, Jakub Konka who resigned from Microsoft to work for the ZSF (https://ziglang.org/news/jakub-konka-hired-full-time/), and whose linker was the first to enable cross-compilation to Apple Silicon, and is already making Rust's cross-compilation just work (https://news.ycombinator.com/item?id=27245369).
Inspiring how such a young language can already be so far ahead when it comes to something as vital as the compile chain.
It's not funny if you understand the relative priorities of the two teams. Rust tends to prioritize working with the system, so all of our default configurations use the system linker. Zig doesn't have the same constraints.
This feature is awesome, and I would love to see it in Rust as well, but like, it's not surprising.
No, Zig absolutely has an awesome compiler. It's not "a toy" or a "one man passion project" and it's condescending to reduce it to those terms, as if it is not also somehow "industrial scale".
To be fair, your comment would also be better served without the insinuation of "just trying to score internet points".
I've made the edit. Thanks for the tip. I've enjoyed and learned from this exchange with you.
Individual insight is often much better than a design by committee, but a language needs a quite robust ecosystem of support from industry bodies at some point to become a viable tool for generic industry scale development.
https://github.com/graydon/rust-prehistory/graphs/contributo...
After the initial commit in the "group" repo, development of the language has switched to a team instead of a single person, with the introduction of additional contributors. This holds even if you only look at the pre-1.0 days which established the core concepts of the Rust language as it stands today (the original language when the team started to work on it was a bit different to 1.0).
https://github.com/rust-lang/rust/graphs/contributors?from=2...
The work since 1.0 can be classified as a massive refinement project, which also does not have a single leader. Zig on the other hand does have a single main contributor/maintainer/creator:
https://github.com/ziglang/zig/graphs/contributors?from=2015...
hostapd is maintained by a single human. They are responsible for about 69% of all commits in the git repo, 55% if you look at the last year. It's still the most widely deployed WiFi implementation of the world. https://www.openhub.net/p/hostapd/contributors/summary
This is not an argument against Zig or that it would not become an industrial scale tool. It just does not feel "lindyproof" yet for real work.
For example NetBSD has Lua in the kernel and FreeBSD has or had a Forth interpreter in the bootloader.
What can be done in eBPF code is somewhat limited, but nowhere nearso as you might imagine from the restrictions imposed on eBPF programs. Things eBPF code is not allowed to do directly are routinely done in functions eBPF is allowed to call, or in the code that calls the eBPF program in a loop.
Overall though, your overarching point still stands that the Rust development is big in that this marks the first time actual Linux kernel code will be written in a language other than C (or platform-specific assembly).
People talk about the insertion process as if it were JIT just because it reuses infrastructure originally invented for JIT.
https://fuchsia.dev/fuchsia-src/development/languages/c-cpp/...
There is a lot more hardware and software now than when Linux got started.
For something different to overcome the initial bootstrapping problem it will need to be significantly better and be pushed by a very large player.
Googles Fuchsia is the only possible current candidate I know of.
The primary goal is to write in-tree drivers in Rust, which can still be loadable modules, but they'll be built to whatever ABI that particular version of the kernel uses. And even for out-of-tree modules, you have to recompile them against the specific kernel and ideally with the same version of the compiler etc.; ABI incompatibilities across C compilers are rare, but not unheard of.
Right, only "repr(C)" types can show up in "extern" APIs. So no data-bearing enums, no Vec<T>, etc.
> Will the gains in safety be contained entirely inside the modules that are written in Rust?
I think for some things the answer is yes, and for other things the answer is no.
If the issue is that callers might retain some pointer or descriptor longer than they should, leading to a use-after-free or something like that, then there's probably no way to fix that with an API-compatible Rust reimplementation. What safe Rust would prefer to do is either 1) force explicit lifetime constraints on the caller or 2) replace the pointer/reference with some sort of smart pointer, which can either keep its referent alive or at least safely detect the fact that the referent is dead. But of course a C-compatible API has no way to represent (1), and (2) would presumably be an incompatible API change, so the Rust reimplementation can't really solve this issue. (Though if both Rust and C APIs get exposed for the new code, then new Rust callers could benefit from (1).)
On the other hand, if the issue is that the caller might pass in some invalid indexes or other metadata, leading to an out-of-bounds array access within the module, Rust can help quite a bit. If the array in question is actually owned by the module (and not just a raw pointer to unknown memory) then bounds checking is required one way or another (unless it's either optimized away or explicitly skipped with unsafe code), and a Rust reimplementation will probably not suffer from out-of-bounds issues. Raw pointers into the caller's memory are harder to check, but even there if they can be converted to a safe slice immediately inside the API boundary, some subset of out-of-bounds accesses can still be caught.
Looking at these two different kinds of bugs, I think it's interesting to think about what "inside the module" means here. In both cases, the UB we want to prevent (either use-after-free, or an out-of-bounds access) is "caused" by the caller's mistakes outside the module, but "occurs" on lines of code inside the module. In the first case, Rust can't protect itself against a bad (C) caller, but in the second case it might be able to.
> You're asking to join us, not the other way around. I'm fine in a world without Rust.
Such a luddite view. "My car gets 40 rods to the hogs head..." I'm always amazed to see such stuck-in-the-mud views from people working in technology which is such a modern fast moving field.
That's something you have to experience first hand to understand, but you know that feeling you get writing C or C++ where new code appears to work the first time you run it? "That's suspicious. It must not have rebuilt, or maybe I forgot to call the new code."
With Rust the compiler can catch so many mistakes that by the time you actually run the code there's a pretty good chance that it is right. My instinct is no longer "suspicious! Where have I gone wrong?"; it's "ha it worked! Though I'd still better double check".
Can we have a compiler flag or a Cargo switch inside the project's file that disables all std API that can panic? Basically complain that the function is not found.
I for one always strive to write my Rust code with `Result<T>` and never rely on panics so I'd love it if that can actually be enforced and checked by the compiler.
But I hope that if it's both possible and not very hard, then it will be considered. I'd personally love to double down on Rust's safety and dial it to eleven in all my hobby and commercial projects.
> [T]he interest in allowing Rust modules into the kernel are a problem for us, due to Rust trademark restrictions which prevent us from applying patches in our distribution without express permission. We patch to remove non-free software, unlicensed files, and enhancements to user-privacy anywhere it is applicable. We also expect our users to be able to re-use our code without any additional restrictions or permission required.
> This is also in part why we use UXP, a fully free browser engine and application toolkit without Rust, for our mail and browser applications.
On the other hand, I continue to be amused by arguments along the lines of "I'd already made up my mind that Rust is bad, but then I actually looked at some Rust code and realized it doesn't use C syntax, and now I'm even more upset"...
Sorry, what does this mean exactly? Could you come up with a hypothetical sentence that is like this? Thanks!
Implications like that may very well be correct, but as always, not sticking to the technical aspects of the subject makes it harder to get anything done.
Rust's tooling isn't quite as mature as it could be, and it achieves its advanced capabilities and performance by eschewing a runtime (which would make it unacceptable) in favour of labourious compilation.
The snark to me seems unnecessary; a more even headed discussion would acknowledge the limitations and focus instead on what can and will be improved or mitigated and what compromises are acceptable.
The tooling will mature with time, panics will be mitigated, and the compile time performance is often a sacrifice worth making for the sake of the safety provided.
You gotta remember that most of the heavy hitters in Linux Kernel development haven't even commented yet. When the Rust side presents something mergeable, the massacre will begin.
There is nothing to comment yet. I've read those driver, it seems unconvincing for me despite me being a rust fanboy. Linus said about panics, I agree completely with him, and with the possibility to get rid of those panics. There is unsafe code where it shouldn't be, for example driver uses unsafe to create and to initialize a semaphore. I believe this unsafety could be removed by using Pin or something like, but when it removed, then it would be the time to comment. And I mean it. Kernel code heavily uses list_head, for example, which is PITA with rust. How to deal with it? I'm not sure that the rust will be better than C, just because it would be rust used in a C style, but it doesn't work. When I met Rust for the first time, I tried it with a C style of designing programs. Many people do, and it just doesn't work.
For now it is impossible to judge how rust will help, we might believe that it will help, because rust is a safety-oriented language, blah-blah-blah. But it is a belief, not something that we could touch, play with, see how it helps and how it obstructs.
I think a lot of Rust people read Linus's "I don't hate it" as explicit approval. But anyone who's followed LKML knows that this is not the case.
Remember what happened with KDBUS? Gregkh fought long and hard to get it merged but in the end it didn't happen because too many people didn't want it.
This is an amazing peoject because it will encourage rust to improve to be better than C for multiple dimensions.
https://en.wikipedia.org/wiki/Servo_(software)
> After Mozilla laid off all Servo developers in 2020, governance of the project was transferred to the Linux Foundation. Development work officially continues at the same GitHub repository, but only volunteers remain, so there has merely been maintenance activity.
Daily usage time of Firefox has actually went up around April 2020, by around 24 minutes and has been at that level since.
https://data.firefox.com/dashboard/user-activity
Also the employees found really good employment at other companies so there seems to be high demand for Rust developers, at least in the USA, and at least for this segment of the market.
Parts of Google Fuchsia are apparently developed in Rust, too.
There are many MCUs that run Linux but definitely are incapable of compiling it.
IIRC most of the build time of a rust program is spent not in the rust compilation proper but in LLVM doing code generation and optimization. I hope both LLVM will become more efficient with time, and rust will learn to pass it such IR that it can process faster. There are obvious incentives for both of these things.
But of course it would be sad if the minimum requirements to compile the kernel grew again, and now excluded older RPis.