Kernel development is hard, and bullshit doesn't go very far in that context. Success for Rust in that environment (with some changes along the way) will be a proof of value.
Pretty sure async traits are coming soon (next couple of versions?) which is pretty speedy considering what I understand to be a semi-thorny problem.
> having a proper concurrency runtime story
Do you mean the language/stdlib shipping an async runtime?
> reducing the reliance on third party crates to ease error handling
I for one don't rely on 3rd party crates for error handling often? Anyhow is the most common one I use, but mostly out of habit and laziness, not actual hard requirement....
If you just mean handling errors from funcs .. it's no different than non-async, no?
No. There are ideas for how to do it, but as far as I know they haven't been tested out yet.
It also relies on pretty large features that are not proposed for stabilization yet (GAT and existential types). When that is done and the implementation strategy is chosen there will also need to be a RFC cycle, a phase of ironing out bugs and finally stabilization.
It'll be ... quite ... a while... I can't see it happening this year.
"I think Rust is a good position in the sense that the language and community has a track record and culture of going and solving problems instead of sitting on them forever. I agree that many challenges of kernel development are going to end up strengthening and evolving Rust as a language."
So I picked some ongoing language examples where this isn't quite true.
For example, suppose you really want Mutex::unlock(). Right now that's behind the feature gate, but it's not controversial, if you feel like this more explicit function call is helpful you could put the work in to stabilize it and get the gate removed in say 1.61.
[[ Mutex::unlock(guard) is just equivalent to drop(guard) since of course dropping the guard will unlock the mutex which is why you usually don't do either, your guard will go out of scope and get dropped automatically, so the reason to want Mutex::unlock() is that it reads more naturally ]]
In C++ the pointer provenance problem has been sat around for almost twenty years as an unresolved defect in, I believe, C++ 98.
In Rust, Aria was like "We should provide a new API to do provenance in a sound way" and she shipped it (admittedly as a nightly feature, not stable) before I'd finished all the prior reading.
Something something let the Java guy cast the first stone.
> reducing the reliance on third party crates to ease error handling does seem to take out forever.
I've written a ton of Rust and I've never used any of these third party error convenience libraries. As far as I'm concerned there is no issue in need of solving around error handling in Rust.
The "please don't offend this language I have intertwined with my own identity with" attitude in modern times is so tragic.
I have always been, and will be, plenty of things, and if it makes you feel good to call me Java guy, when someone criticizes your beloved Rust, please do.
But you're over-reading into my defense of Rust. My point is that the specific things you criticized as taking a long time for Rust are not actually taking a long time. My point with poking Java, specifically, is that Java is going on 30 years old and doesn't yet have an analogous feature. Moreover, since we're talking about Kernel dev, do C or C++ have a standard async runtime? Considering that there's little-to-no prior art for doing the kind of async API Rust wants without active garbage collection, I'd assert that it's hard to criticize how long it's taking. Do you know how long it "should" take? If so, how?
Even if C is not my primary language of choice, i would definitely try adapt myself to the ecosystem and not the other way around.
You have all the knowledge of other peers, manual, books, all the libraries, the whole ecosystem.. this cant be replaced.
Also there's something else about C nowadays, is the lingua-franca, the latim (or english) of programming languages. We use it to expose api's to others in any other language that want to consume it as a library.
There's something about culture that people often forget in tech.. it's the real backbone of any project that it's on its own feet.. and when you want to enter in a community you will be better of if you learn and adapt yourself into this community culture instead of creating cultural clashes into the community and try to overtake it (be it hostile or not).
People should be aware that this effort will make it possible to create rust-based kernel drivers and that's it. the RIIR folks are delusional and hype fueled and its better if the sane Rust community get away from them or start to get them back into reality as i bet they are not willing to expend 10 or 15 years of their lives rewriting big and complex piece of software for a likely no return as people will tend to keep using the software the have more community and that are stronger.
It's a much better approach for Rust or any other programming language to become research darlings and eventually become the primary ecosystem of a research OS that went well and is the thing that will replace Linux. The language alone wont do it, it must be able to be a contender to UNIX and POSIX, and whatever language that is in such a system will probably be the one that will become the dominant one in such a ecosystem.
Also another good approach is to virtualize the Linux Api like gvisor does is userspace or the fuchsia OS(and even FreeBSD) does in the kernelspace. So that you can create your OS and kernel in the best way you can looking ahead, and have this Linux compat layer where applications dont even need to be aware they are not actually running in Linux.
> * 11 hours ago | parent | prev | next [–] > Rust will quickly replace C in the kernel, I have no doubt about it
I've have seen tons of such a comment in all sort of products when the matter is discussed around here and elsewhere (twitter, reddit, you name it)
If you want to really get serious about this, i can feed this comment sections with tons of evidence over the course of the years.
I suspect that this "RIIR" that you seem to believe is some kind of "movement" is just a random assortment of clueless people posting in random places.
A simple pattern that rulers to turn a blind eye to the connection between belief and action.
Yes, i'm sure there's an imaginary opponent downvoting my comment.
Also there's a lot into my comment, and people are not even noticing it in the whole context.
If there's no reason, why people get so upset? just move on if you are not being mentioned, as it will clearly be the case if i'm talking about "imaginary opponents"..
> The one proposing it is a long-time kernel contributor.
I'm not saying anything about people working on it specifically, if you read my comment, there's a clear separation between serious people and the hype crowd (which is not just RIIR, but now also cryptocurrency fellows, etc).. i can't say where the people working on this fits, i don't know them. Don't know from which part of my comment you took that conclusion.
There are quiet, clever, serious people doing the work, like Hoare, Matsakis and the people that are real enginners, i have all the respect for them (and im pretty sure Rust have tons of such a people). To be fair, all languages have all kinds of people but i don't know what happen to some of them that tend to attract a certain type of people more than others, like the feeling a got from Haskell community more often than others (but given the community was much smaller)..
For instance talking about culture, C succeeded exactly because it was a pragmatic language very simple and efficient like their founders to get things done. With this culture, things happened to be done around the language and we have the ecosystem we have today.. it's a great hacker spirit of more humble, hard-working, behind-the-cameras sort of people which i sincerely miss in the days of instagram, tick-tock and tech celebritism.
I don't know what it is about Linux that makes people say this. Kernel development has most of the same constraints as any other embedded context, which Rust has plenty of focus on. No it's not as mature as C, but few languages are.
Plus if you go looking in the kernel you can still find plenty of bullshit hacky code. It's not special, it's just another random open source software. The quality very much depends on the individual maintainer of that subsystem and how much has been invested in that area.
you think that might do it?
I'm tired of having to deal with this culture war crap in our profession. These languages are tools. C, C++, and Rust all compile down to the same LLVM IR (or GCC if you stray from rustc). There are certainly semantic and grammatical peculiarities that affect how each of them do so[0], but by and large running a simple Rust program and a simple C program through Godbolt will do a lot to disabuse you of the idea that the two are irreconcilably different.
To anyone else who wants to write performant Rust, my advice: (1) no_std, if only to focus the mind, (2) .try_foo(), not .foo(), b/c allocation is fallible, (3) always set `opt-level` to at least 1 (1 is far further from 0 than 3 is from 1, ime), (4) use stack-allocated alternatives to heap-allocated types ('smallvec' or equiv vs Vec, 'smolstr' vs String, &c) even at the expense of overallocating buffer space, (5) exploit vectorisation where possible (e.g. SIMD), in general practising mechanical sympathy, and (5) parallelism is not a panacea, whereas cache locality usually is. Measure everything, but also: memorise every instruction and how many cycles it takes, and think in those terms - in terms of your assembled, perhaps-handmodified code - rather than unscientific laptop benchmarks. (Jeff Dean's famous 'numbers every programmer should know' are a good start but are just the very basics, and obv his exact values are long obsolete, in some areas [disk] more than others [CPU].)
[0] These are discussed extremely soberly and intelligently here, for you or anyone else who may be interested: https://kornel.ski/rust-c-speed
Today Rust does not overlap Linux in terms of platform support. There are (small but very much alive) communities doing Linux on architectures that Rust has no support for and in some cases has no plans ever to support. So this makes drivers the only case where choosing Rust doesn't mean some people lose out, as a platform e.g. with no PCI bus doesn't get to run PCI drivers even if they were written in C.
I expect that over the next say, five to ten years, two things will happen to greatly improve this, maybe to the point where you absolutely could rewrite core Linux code in Rust if you wanted to. Firstly, Rust will get more platform support. Linux doesn't really need Rust's "Tier 1" (Linus doesn't check every kernel release passes tests on all real Linux target hardware as I understand it) but clearly you want Rust to at least build and take patches for every Linux platform some day. Secondly, some older platforms will "rust out". If your community is nursing 30+ year old hardware and increasingly more maintenance work is shared between fewer shoulders at some point "Linux-next" is not a priority and your platform will stop being supported while effort moves to exciting new hardware.
Nit, I believe actually the Rust and Linux platform sets would be considered "overlapping sets" in the mathematical sense :), since neither is a subset of the other.
e.g. Rust platforms include things like the NetBSD Rump Kernel and Redox and I think one would be hard-pressed to claim that Linux supports those as platforms.
I'm interested in kernel development and I like the idea of working on it for a living. Can you give more details about your job? What does it consist in? Is it mostly code-review? Or are you responsible for maintaining a part of the kernel. Who is the entity that pays you, and what are the criteria they'd use to pay a new contributor to work on kernel full-time? Finally, can you point me to beginner-friendly things to work on to get started? how do I know which part of the kernel I should study and contribute to?
Hence the crazy idea of now using WebAssembly in Firefox modules instead.
Most of that code is surely related to what was replaced initially instead of new subsystems being ported.
Irrelevant to the question of how much Rust is in Firefox.
> is surely
Okay, it's clear you're not going to change your mind, no matter what.
Not to mention how much less experience you need in Rust to not blow everybody's feet off by mistake. I reckon if you have 10 years C++ and six months Rust, any Rust you write is already more likely to deliver reasonable performance without setting everything on fire than your C++. Because of the constant exposure to outright malevolent stinking garbage (in the form of other people's HTML, CSS and Javascript) the browser needs to be exceptionally robust, and C++ just isn't very good for that. So Rust is often a better fit for what Mozilla do.
Chrome folks have been playing with Rust, but seem more keen in improving their C++ static analysis tooling instead.
As for Mozilla, it is 10% of Rust code and lets see for how long Firefox still matters, given the existing 3% market, even EdgeChrome has surpassed it.
I would say that at this point that's good money after bad. Linus of course also put a bunch of effort into static analysis, that's what "sparse" is.
The thing you run into immediately is that your programming language doesn't express the thing you wanted to analyse very well. So you have to annotate your software (Linux is sprinkled with sparse annotations), and now you've added an extra opportunity for mistakes, because the annotations are transparent to the compiler, so you can write code which analyses as correct but compiles to something incorrect. "Hooray".
What Mozilla did do was lay off many of the people working on Rust itself as their full-time job, as opposed to people who use Rust to do their work at Mozilla. And the Servo team, unfortunately.
The 10% as pointed out in a sibling comment, while admirable isn't large parts.
As you can see from the breakdown someone posted [0], almost two-thirds of the Firefox codebase is HTML / JavaScript / Python (for tests) / assembly / Java (for Android), none of which is a candidate for being rewritten in Rust to begin with.
If you just look at the portion written in native system languages, Rust is slightly more than 1/3 of that code already and still climbing.
Probably less than 30 people in the world understand the thing as a holistic entity. From evolutionary terms, there's a giant risk of losing relevant technical knowledge to keep it up in the future.
I do not support adding it to the Kernel, I think we should just throw away the kernel entirely, but I understand why they're looking at Rust.
At no point did any active kernel developer express any such sentiment. Nobody is worried about not being unable to attract new blood.
As we only have two other "big" (popular) kernels to compare to this one, do you think they (Apple and Microsoft) have more people "holistically" understanding the entire kernel or less? Since the nature of those companies are closed-source, I'm fairly certain even less people understand those ones "holistically".
> I do not support adding it to the Kernel, I think we should just throw away the kernel entirely, but I understand why they're looking at Rust.
How would that work in reality? Re-use the existing tests to build a new kernel from scratch? Sounds like a very far-out idea that wouldn't help with any of the current problems, but I'm happy to entertain the idea and hear your reasoning here.
While I would tend to agree that a full production replacement would be such a massive undertaking as to be impractical, https://github.com/nuta/kerla does something very like that - Linux userspace ABI on an all-new Rust kernel. (And even at this small scale, I find it mind-blowing that this worked)
Fortunately for the kernel, despite being niche, it has a rock-solid onramp forcing people to get into it. There will always be companies interested in the n'th degree of performance, both generally, and for some specific hardware. Someone has to go do the relevant kernel work for those things. So while you or I may never touch it, it is effectively impossible for a kernel like Linux to just rot away because nobody cares. It would require first a multi-year, if not multi-decade process of fading first.
It's on most of the smartphones on the planet. Plus it is the dominant server OS. Plus in the top 3 in many embedded categories (a highly diverse set of technologies)
Sadly enough, probably not for long: https://en.wikipedia.org/wiki/Fuchsia_(operating_system)
I’m not saying that Fuchsia is necessarily bad, I’m saying that Google will do anything to get away from GPL code, including, if necessary, forcing Android to Fuchsia. It doesn’t actually matter if Fuchsia is any good.
Why not? Certain people are using microkernels, for christsakes. Why not Fuschia on the server?
> Google, however, does not control what runs on servers, and server hardware manufacturers know that if they don’t have a driver in Linux, they won’t sell very much of their hardware, since existing hardware run Linux-based systems.
I'm not sure if "It's the way things are" is quite the argument its made out to be. Some server hardware is beginning to look more and more like a phone/Chromebook. And things do change. "Not in a million years" was the argument made against Linux compared to the traditional enterprise UNIX vendors and where are they now?
If there really was enough reason and will to do it, they would just track down those people, or they would remove that code and replace it with something else. Just like they've done every time in the past when there was copyright problems. Just like any other big open source project has done when licensing became a problem.
Anyway, what Linux developers might be afraid of can not be mitigated by simply switching to an MIT license. What would happen is basically some modern variant of this:
That's a pretty wild assertion and files in the face of the highly active kernel development process.
e.g. > Linus has released 5.18-rc1 and closed the merge window for the 5.18 release... 13,207 non-merge changesets were merged during this merge window.
https://lwn.net/Articles/890119/
> I think we should just throw away the kernel entirely
That's the dumbest thing I've see on HN in some months. The kernel is deployed in hundreds of millions of devices worldwide and continues to be the dominant OS in many many sectors.
There are billions of Android phones alone. Not to mention the huge numbers of servers, IoT devices, embedded computers, and all the SBCs in my closet.
"The timeline for shifting to an "upstream first" cycle for new features starts in 2023, with 2020-2022 dedicated to making it work for pre-existing functionality. The Pixel 6 is expected to be the first Android device to ship with the GKI and Linux kernel 5.10, marking a major step in this process."
Means this is still one year away, if it actually happens, and then only Pixel devices will support it anyway, as so far most OEMs couldn't care less.
Not to mention that upstream will never accept all the things that make Android Linux not really Linux.
No, OP is right. 99.9% of things added to the kernel in any given release are drivers (mostly for obscure server hardware that the average LKML lurker would never have access to). So while there's certainly active development, it's all centered around drivers, and the "kernel" part of the kernel are mostly stable (modulo the occasional greenfield project like eBPF). (Note that I said 99.9% of things added, I'm sure security patches make up a lot more than 0.01% of kernel patches).
I guess I need to go let them know they are Nobodies. oxff said so.
What would you use instead?
HaikuOS, SerenityOS, Redox, and ReactOS are all going strong. The BSDs continue to advance as well. Redox is written in Rust even.
I believe Google sees Fuschia as a true Linux competitor.
When you say “we should just throw away the kernel entirely”, what are you suggesting?
That's simply false. I'd love to work on the kernel. I have immense respect for the people who work on it. Only reason I haven't tried contributing code is I don't think I'm skilled enough.
Probably a lot more than 30. There are probably more than 30 PhD students studying kernels at this very minute. However, I think you're right that no one is interested. People greatly underestimate the effect the kernel has on the full OS. They think it's "just" the kernel. This is why so many people keep saying that Windows will get a Linux kernel in Windows 10, no, 11, no, 12, etc. It's just a "kernel", like a grain of corn, something insignificant. So there isn't much interest in working on kernels. It's as unsexy as it gets.