Rust support in the Linux kernel
lkml.org
lkml.org
>A few notes for people who haven't been tracking the Rust-in-kernel situation. Drivers especially have been targeted for a few reasons. First, drivers are isolated, with abstractions that don't require interaction with the rest of the system. Second, Linux is available on more systems than the Rust compiler has been ported to. Because drivers are inherently system-dependent, it is not an issue to introduce a dependency on rustc on compatible systems. Third, bugs tend to arise more in drivers than other parts of the codebase, so Rust can do the most good starting there.
[1] https://www.reddit.com/r/rust/comments/rb50yn/comment/hnmshs...
This means that using it for the core parts may be feasible sooner rather than later, no? Developing those abstractions may not be a priority of course, but it might be allowable at least from a platform support perspective.
Linux's own support for the long tail of more obscure platforms is also an ongoing process. People aren't asking for Rust to do better than the Linux kernel in that regard, they're just asking for it to not immediately disqualify a range of platforms.
One could reasonably argue about the value of supporting those platforms, but the Linux kernel, reasonably, doesn't want to make that decision on the sole basis of the availability of Rust support.
> And for exceptionally weird platforms (e.g. where a byte isn't 8 bits) Rust may never bother to contort itself enough to support those, although perhaps Linux also doesn't support such platforms?
Linux doesn't support such platforms, and I don't think Rust ever will either.
Almost all software not written especially for such devices doesn't support non-8-platforms. Heck, POSIX mandates that char is 8 bits.
This is partially what caused the shitstorm when the python Cryptography library took a hard dependency on rust to compile:
You can still run 31-bit s390 usermode binaries on an s390x kernel, but that's a different beast.
Ah thanks, I didn't realize that was possible. I'm no where near rich and/or old enough to have ever encountered an s390 machine IRL :)
More like much later. There are several performance-critical patterns used in the Linux kernel that are difficult to express in unsafe Rust and impossible to express in safe Rust.
See https://paulmck.livejournal.com/62436.html and especially https://paulmck.livejournal.com/64209.html
I think promoting the use of Rust for driver code is an excellent compromise.
As an aside, I'd like to see Linux go further and move many of these drivers to user space. Now that we have io_uring, old performance-based arguments for keeping certain drivers in ring 0 hold less weight.
There's no specific goal to support every platform Linux supports. We have four tiers of support: 1-3, and then not supported in-tree. The project itself only moves things forward on a given platform if there are people championing that platform's support. If folks want to get Rust in more than just drivers and into the kernel proper, and platform support is an issue, then it's up to them to drive the platform support. For example, aarch64-unknown-linux-gnu only recently gained Tier 1 support, thanks to Arm themselves driving the effort.
For the details on what is supported today, and the process to expand that list: https://doc.rust-lang.org/stable/rustc/platform-support.html Here's an example of a contributor adding a new platform at tier 3: https://github.com/rust-lang/rust/pull/79608
the days that linux was only used as a secure server OS are long gone. it is used everywhere for all kinds of applications and by lots of small manufacturers who are using the pi or other platforms.
also, you cannot just develop a driver in isolation, you have to take the typical use cases in mind while you make choices how to handle things.
So as long as it will be possible to use the system's llvm+rust to build for all 4 CPU archs (at once) I'm currently building my kernels for, I'll probably not be too cranky about rust in the kernel.
Plus it's not like most people are going to dive into driver code without experience in the driver itself. So it's a niche set of folks this would affect, the majority of whom would likely be writing the rust code anyway since they're likely the ones most familiar with the driver aspects.
I ask you to imagine a situation where a kernel is written in assembly, and they decide to start allowing C to be used for drivers. How would you view a change such as that?
> C is a simple language, the complexity comes from large systems, while Rust is just not.
I mean I agree that C is simple if you're making a 50 line program that writes to stdout or something. But more often than not with C or Rust you are working with a large complex system. And I'd argue that when working with such a large system, Rust programs are less complex than C programs.
C makes many questions when reading trivial to answer, questions like "how do I get here?", "where does it go?", "what is this type?" and "what does this code do?" are generally very easy to answer without expert-level C knowledge. From what I've seen, that's not true for rust.
If you didn't know how to code in assembly already, you now know enough to do everything C can do with a simple reference sheet. All the extra boilerplate (such as program segments and where to store costants such as what the 'strings' program sees in a binary) you can read two to three pages to explain what you need for whatever OS you're on.
Assembly isn't hard, it's just tedious.
GCC's cross-compilation story is indeed a nightmare. I expect if you have to use the GCC Rust backend (after it's ready) for arches rustc/LLVM don't support, you'll have that pain, but I expect it will just be a matter of adding an "--enable-language rust" to the GCC configure script run, similar to adding C++, ObjC, etc. to a GCC build. So the cross-compiler build will take longer, but won't involve extra steps or infrastructure.
Then you have a cross-compiler in some --prefix that you can point the kernel build system to via some environment variables and you're done. It's a well known process.
It's basically built just like any other dependency and doesn't take much time.
Hardly a nightmare. If it would be possible to just add rust to the list of gcc languages, that would indeed be very easy.
The main drawback of Rust here I think is the longer compile times (I dread to imagine how long a fully Rustified kernel would take to build) and a generally more complex, but also more expressive language.
More generally I trust the kernel maintainers not to succumb to the hype and overdo it. People who still exchange patchsets over mailing lists are probably unlikely to be blinded by shiny things.
one of the biggest hurdles for me was the DTB declarative language. that was the most stupid idea ever to invent that and i lost 100s of hours trying to get my DTB files right for my hardware. so you can understand i don’t want more hipster shit to deal with.
Also at this point, Rust isn't "hipster shit". You're just not familiar with it, and that's fine. But it's in wide use now and proven to solve major classes of recurring issues.
Yes. Rust helps with that because as a language it simplifies avoiding huge classes of vulnerabilities.
Rust simplifies it for the developers by moving part of the mental model of the developer into the compiler.
Of course you need to know what you are doing still, but pretending humans don't make mistakes even when they know what they are doing is ignoring reality.
And those drivers can continue to be written in C. Rust is merely an option.
I'm sure it gets better as you memorize the millions of different types, but what benefit do you get from all this pain?
I also think it's just a boy's club at this point. Big pipe dreams of rewriting all the C++ and C code is all we here about. It's not going to happen. The language is too f*k*g hard to learn so no one will. It's like trying to make everything in Haskell, c'mon, get real.
Someone had to say it...
I don't think I've experienced this in any language, even ones with the most expressive type systems. More typing is better, as far as I have experienced, and as far as I have heard from people who take it up in their projects.
The compiler will tell you exactly what goes wrong if you pass the wrong type. If you mean that type inference was confusing, it's the same as `auto` in C++. Except the Rust compiler does an enormous amount of handholding with errors, often showing you exactly how to correct your code.
Probably mostly because of that.
Not that the other points are necessarily easy to follow since they're not well explained. If knowledge of why some type was hard to know or had to be googled (or for example noting what one of those types is) was presented, then people might have been able to respond to those points.
Instead it reads as "it's hard, I used it and it's hard, and I had to do some stuff I didn't like, and so my impression is it's just a bunch of people faffing off and it's too hard to take seriously", which isn't really useful for a discussion, and additionally is dismissive. Every one of the criticisms presented could be said about C without any change because no actual concrete details are provided.
It's not like Rust is perfect, there are many valid criticisms of it that have real examples and make useful points. Those comments probably deserve your defense more than this one.
No matter your skill level, that won't stop you from committing memory-safety mistakes that may lurk for years until they suddenly result in a catastrophic failure. Evidence for that is abundant.
Sometimes I feel like C's compiler is almost adversarial, going out of its way to find small mistakes in my code and have them blow in my face at runtime instead of notifying me. I understand why it works that way of course, but usually when my Rust code compiles, it works. I can't say as much for my C code, despite nearly 20 years of practice.
I do share some of your frustrations with the DTB thing, I'm not a fan of it either, although something like that was needed and I'm sure any alternative would have its haters.
From the perspective of people who don't like aspects of Rust, it's a yucky goo.
This is _not_ like with C. Few people are enamored with C, but - the language itself is relatively simple (Duff's device notwithstanding...), so in my analogy it's like water. You may not adore it, but you can see through it, and you'll be ok with gulping it down to quench your thirst.
I beg to differ. This is exactly how I feel about C. C is the language that's confusing as hell to read for anything other than really simple use cases AND makes it super easy for me to shoot myself in the foot.
Know thy standard and thy compiler optmizations, least you be surprised by the outcome.
This is also possible for C. Most commonly around Undefined Behaviour. C only does what you expect it to do if you know the C spec inside out. Rust can have confusing behaviour too, but it is generally much better in this regard than C or C++.
You may find it easier to reason about C than Rust, but that's only because you know C better. I find it much easier to reason about Rust.
if you don't use pointers, yeah. but like you can't built anything useful without pointers.
It's a question of whether unguarded pointers (and a lack of language features that keep pointer math from devolving into an unreadable mess) are good to have in areas where pointer errors can result in critical bugs. We started discouraging goto statements not because developers didn't understand them, but because they did understand them, and they understood that the tools you use influence the type of code you write. And Rust doesn't even really get rid of pointers, it just adds rules around them.
We all accept some degree of abstraction/safety tradeoff, otherwise we'd program in lower-level languages than C that are even simpler and even easier to understand. C does introduce a lot of complexity over more primitive assembly languages. And Rust introduces a lot of complexity over C.
Different people pick different points where they think the tradeoffs are optimized. But we have a pretty large amount of data suggesting that even <quote>good</quote> programmers in industry make pointer errors in C. So it's pretty obviously not lack of understanding that's causing these issues, it's that unrestricted memory access and pointer math is inherently error-prone. Whether it's SO error-prone that's it's worth introducing a much more complicated, rigid language into the kernel is left as an exercise for the reader. There is a downside to introducing more complexity here, all abstractions have downsides.
But it's real dismissive to say that if everyone understood C pointers then their code wouldn't ever devolve into an unreadable mess. I just don't think that's an opinion that can be backed up by any real evidence, pretty much every org struggles to some extent with writing secure C code. And we have a lot of experience in the field of software engineering that should have taught us by now that sometimes unreadable code is the result of language affordances and patterns.
My wild guess is that most "simple" pointer errors (use-after-free, double-free, unintentionally using unitialized memory, ...) happens at interfaces where there isn't perfect comunication about what component is responsible for keeping track of each task.
> "Most bugs are a result of the execution state not being exactly what you think it is."
There is an aspect of language design that is about making it hard to be surprised by things. And I vaguely suspect that a lot of pointer errors fall into this category; people assume something is being freed, they assume that a subroutine is cleaning up after itself or that it's not manipulating something, or that it doesn't need to call some setup method before it's run. These are boundary errors between interfaces where allowing users to (inadvertently) mask what is happening can lead to logic that feels very convoluted and hard to follow, and can also lead to simple errors around just forgetting to do stuff.
I wouldn't call that Carmack quote a mantra, I'm not saying it explains all programming (or that all architectural restrictions are good), it's just a lens that I keep coming back to when I think about language design and architecture; there's a real tradeoff in removing flexibility, but there is also a huge amount of value in systems/patterns that make it very hard to bury relevant information. And if a language can auto-surface some of those problems, that can lead to cleaner code.
The time scales with the compiled unit size. If you're compiling drivers/modules, they're going to be fairly small and not much slower than C.
the lack of memory safe languages is probably the nr. 1 contributor to critical bugs in software and regardless of what we're working on we ought to learn new languages if they help to improve software quality.
Let me explain why I think that:
In the short term, having more eyes looking at and re-implementing the scaffolding for drivers and the drivers themselves will certainly lead to a very positive outcome - many potential bugs (of which, most of them will not really be exploitable, but some will) will be uncovered which couldn't have been introduced with safe Rust. However, at some point, the coolness factor will drop, drivers will become fossilized, a larger developer base that has to maintain older Rust drivers might cut corners, use more unsafe features, misunderstand/misremember the nuances of this or that, and what was once exciting is now tedious. Maybe not, maybe Rust will slowly replace the whole kernel, however, at the current stage at which Linux is, I find it unlikely (but possible).
The great thing about Rust is that it makes it much harder to cut corners, and much more obvious when someone does (making it easier to pick up at review stage). Invariants can be built into APIs in a way that simply can't be done in C.
The machine doesn't get bored. The fact Rust may not be "cool" any more doesn't alter the fact that your subsystem maintainer isn't going to accept your patch that doesn't compile because you tried to take the lock for foo, but then used locked variable bar - something C has no problem with but which in Rust of course does not compile.
Regardless, if you want to keep using C to write your Linux drivers, the new Rust support does not change that.
One of those toolchains is used to build the bootloader, kernel and majority of userspace software for the devices. The other toolchain is used to build an I2C temperature sensor driver that was rewritten in Rust so someone could put Rust driver experience on their resume.
Dismissing this and saying "suck it up, this is the price of progress" is fine. But it doesn't change the fact that this could be a giant PITA for people making devices and developing the BSPs for them while providing almost no value. I'm not trying to argue that appeasing this group is important enough to totally reject Rust from the kernel, but as a maintainer, making life difficult for a huge number of kernel users is probably something you'd like to avoid.
I think a lot of this will come down to how Rust patches are reviewed and accepted. If Amazon or Facebook want to upstream some new Rust driver they wrote for their own hardware, there's probably no negative impact to anyone by merging it. But I'd expect maintainers will be very hesitant to accept patches that touch existing drivers. I guess time will tell.
Most of "tuning the kernel for specific applications" consists of making changes to the kernel config or compiler settings. Rust won't change the kernel config system, and compiler settings for Rust are quite similar to those for C, as Rust uses LLVM as its backend, and LLVM is used by Clang which goes to a lot of effort to provide a mostly GCC compatible interface.
> in the end, you still need to understand what you are doing, and rust is not going to change that.
If think you understand what you are doing in C, you probably don't. C is a fantastically difficult language to write correct code in at scale; even the best developers make simple mistakes all the time.
I understand the frustration at seeing something else you might have to learn and deal with that might make things a bit trickier during a transition, but that's pretty much the nature of keeping up with any kind of ongoing development process. It's not like the Linux kernel sprang fully formed out of Linus's head on day one and it's just been a matter of adding drivers and making small incremental fixes. There are all kinds of changes that require you to change and adapt, like the removal of the Big Kernel Lock, or pretty much any internal API work, and yeah there can be work to keep up, but that doesn't mean you should hold back progress.
I have no idea what this is supposed to mean. Drivers obviously interact with each other and with the rest of the system via all kinds of in-kernel interfaces.
Drivers are also not inherently system dependent. Even the sample rust driver is for android binder, which is not a real HW device, but some kind of IPC mechanism. So that one is quite system independent.
So if you port some core functionality of the kernel to rust, that would make rust a hard dependency for the whole kernel.
But the total footprint of the interfaces that typical drivers use is fairly small. When a rusty shim is provided that provides access to those interfaces, you can write pure rust drivers.
For example look at typical audio codec drivers. They barely have any procedural code (and when they do it's just some form of register value mapping anyway), locking/concurency and almost all resource management is handled at the higher level and the driver is not exposed to it.
Another kind of a driver that features a very limited dependency on Linux interfaces is a SoC clock/reset driver. And clock drivers are even more boring in this aspect. Code vs data ratio is probably < 0.1 in many of those.
I guess some drivers that do more complicated stuff, and deal with concurrency issues or more complicated resource management may benefit, but they'll likely need to touch much larger API surface, too.
And I dunno, even the sample driver in the patchset has things like this:
let mut data = kernel::new_device_data!(
gpio::RegistrationWithIrqChip::new(),
PL061Resources {
// SAFETY: This device doesn't support DMA.
base: unsafe { IoMem::try_new(res)? },
parent_irq: irq,
},
// SAFETY: We call `spinlock_init` below.
unsafe { SpinLock::new(PL061Data::default()) },
"PL061::Registrations"
)?;
// SAFETY: General part of the data is pinned when `data` is.
let gen = unsafe { data.as_mut().map_unchecked_mut(|d| &mut **d) };
kernel::spinlock_init!(gen, "PL061::General");
let data = Ref::<DeviceData>::from(data);
When I look at the kernel's spinlock type https://elixir.bootlin.com/linux/latest/source/include/linux... that doesn't really help me determine why whatever this `gen` [variable?] is needs to be passed to `spinlock_init`.
And what's with the two `let data`. Are they the same type or is the type now different and the first data variable now hidden? Because it looks like second `let data` gets somehow transformed from the first `data` into something else. So now we have two identically named `data` variables with different types in the same function?
Wasn't this supposed to be some simpler language to draw more people to the kernel development? At least with C each variable has a type that's quickly identifiable.
I guess after 15 minutes I see what this is trying to do. It passes some newly created spinlock object to `new_device_data` function and then calls `spinlock_init` on some part of the data returned from `new_device_data` that needs to be extracted from it via some really obtuse transformation, just to initialize the spinlock later instead of directly in `SpinLock::new` for some obscure reason, using two unsafe {} blocks in the process. But oh my god, if this kind of obfuscation is going to be the new standard for the kernel,... At least C is the devil I know. :)
If you want to practice, check out https://github.com/immunant/ioq3/blob/transpiled/quake3-rs/q... or whatever.
You are judging Rust by looking at what idiomatic C code looks like in Rust.
That said, idiomatic Rust is difficult for me to read/understand as well. I do not expect device drivers in Rust to be readable (by me), and if they have lots of unsafe blocks then we are back to square one I would say.
pub static mut g_color_table: [crate::src::qcommon::q_shared::vec4_t; 8] = [
[0f32, 0f32, 0f32, 1f32],
[1f32, 0f32, 0f32, 1f32],
[0f32, 1f32, 0f32, 1f32],
[1f32, 1f32, 0f32, 1f32],
[0f32, 0f32, 1f32, 1f32],
[0f32, 1f32, 1f32, 1f32],
[1f32, 0f32, 1f32, 1f32],
[1f32, 1f32, 1f32, 1f32],
];
pub static mut g_color_table: [[f32; 4]; 8] = [
[0, 0, 0, 1],
[1, 0, 0, 1],
[0, 1, 0, 1],
[1, 1, 0, 1],
[0, 0, 1, 1],
[0, 1, 1, 1],
[1, 0, 1, 1],
[1, 1, 1, 1],
];
If you are interested in looking what Rust written from the ground up in the same domain looks like, you can maybe look at Bevy[1].
The biggest source of "noise" in Rust is explicit type conversions, FP constructs and longer types (`.into()`/`.as_mut_ref()`/`.map()`/`Result<Option<T>, E>`).
[1]: https://github.com/bevyengine/bevy/blob/main/crates/bevy_ren...
BufferInfo {
buffer_usage: BufferUsage::VERTEX,
..Default::default()
},
impl<T> Default for Option<T> {
fn default() -> Option<T> {
None
}
}
The .. is the spread operator, and it takes every field of the value on the right and applies it to every field of the struct constructor on the left. This only works for the same type (it's not only looking at the field names).
`Default::default()` is a qualified call on the trait equivalent to `<_ as Default>::default()`, where `_` is inferred from context to be `BufferInfo`.
What that code is doing is saying "build a default BufferInfo with field buffer_usage set to VERTEX".
The spread operator has a limitation you might have intuited from the description that it can't work for types where some field(s) should always be set but not others. To get around that you can use the builder pattern
BufferInfo::with_buffer_usage(BufferUsage::VERTEX).build()
struct Foo {
field: i32 = 42,
}
let foo = Foo { .. };
Specifically, when you initialise a structure like this, the .. is "struct update syntax" and is followed by an instance of that structure from which remaining fields will be copied. Default is a Rust trait (similar to an interface if you've never seen Rust traits before) and it has the method default() which gives you a default of whatever type.
Rust has type inference, if you don't specify a type, but there is only one possibility that could work, the type inference might (and in this case does) see that and you needn't bother laboriously writing down the type. This can only happen where there was no possible ambiguity. In this case, the only type you can use for struct update syntax is the structure type you're updating of course, BufferInfo, and so Default::default() must mean the default BufferInfo.
If you found the definition of BufferInfo it probably says #[derive(Default)] which means, using a macro just implement Default for this structure in the obvious way, although it might actually have an explicit Default written out instead.
Where do you see hacks being used in, say, the refactored code? If you meant generally, then sure, they do.
// SAFETY: This device doesn't support DMA.
let base = unsafe { IoMem::try_new(res)? };
let mut data = new_device_data!(
RegistrationWithIrqChip::new(),
PL061Resources { base, parent_irq: irq },
// SAFETY: We call `spinlock_init` below.
unsafe { SpinLock::new(PL061Data::default()) },
"PL061::Registrations"
)?;
// SAFETY: General part of the data is pinned when `data` is.
let gen = unsafe { data.as_mut().map_unchecked_mut(|d| &mut **d) };
spinlock_init!(gen, "PL061::General");
let data: Ref<DeviceData> = data.into();
Ok(data.into())Most Rust is written with the implicit assumption that it will be read in an IDE that displays types, so explicit type annotations are usually kept to a minimum. Doesn't always translate well outside of the IDE, so I wouldn't be surprised if the consensus for kernel code shifts to more annotations.
> When I look at the kernel's spinlock type [...] that doesn't really help me determine why whatever this `gen` [variable?] is needs to be passed to `spinlock_init`
The rust version of the documentation makes an attempt at explaining it, though honestly it could be a lot clearer https://rust-for-linux.github.io/docs/kernel/sync/struct.Spi...
> And what's with the two `let data`.
That's pretty ideomatic in rust, probably borrowed from functional languages. The new definition hides the previous one, and the implication behind reusing the name is that it's fundamentally the same data, just with a different type, or different mutability (here it was converted from mutable DeviceData to an immutable Ref with the same DeviceData).
Now, some of that is just a matter of learning curve; once you have learned the Rust-in-Linux abstractions the above will probably be somewhat more readable.
Coming from someone who's more familiar with Rust, and less so with the kernel, here are a few notes.
* It might not be the best idea to post the example drivers with inferred types, for this familiarization reason. While there are a few types (such as those involving closures) that you can't write and can only use via type inference, or others that are too large to want to write out, these types should be fairly easy to write and it would improve readability during the familiarization process
* A lot of these things that you were having trouble with look like they are basically just dereferencing and accessing some data contained within the spinlock, and casting what is only guaranteed by the API to be an immutable (shared) reference into a mutable (unique) reference in an `unsafe` block. This is a place where trying to use simple Rust wrappers over core kernel structures which weren't designed for Rust makes things a bit more difficult to read, as you need to do a bit of an unidiomatic dance to tell the compiler "yes, the language and type don't gurantee uniqueness, but I promise this reference is unique and I won't misuse it."
* One of the small little humps to get over when learning Rust is to learn that "mut" actually means "unique", and unmarked (immutable) references means "potentially shared." But since you do need to sometimes write to objects that are potentially shared, it just means that you need to either guarantee yourself or use some kind of synchronization mechanism to ensure that at any given moment of time that data is only mutable in one place at a time.
So, I don't know the details of these APIs myself, but when I look at this here's what I see
* we're constructing some device data object protected by a spinlock
* we're providing some brief justification for why this particular usage is safe, since the API and compiler can't guarantee that it's safe
* we're extracting a mutable reference from it, for which we need to uphold the single exclusive access principle without assistance from the compiler
* we're wrapping that in a new reference to device data type which presumably provides a more idiomatic interface than what the raw primitives provide
I'm guessing that the following code which actually uses this Ref::<DeviceData> object will then be a bit more readable and require fewer uses of unsafe, as it looks like this code is what is providing that translation between some core kernel primitives which are not terribly idiomatic to use in Rust and a safer, more idiomatic wrapper.
a `cargo fmt` option to add inferred types to the source code would have some advantages for that.
Consider this: Does your GPU need to talk to your mouse or your real-time clock? Or does your sound chipset need to talk to your HDD? The answer to both should be no. Even a file system driver does not need to know about the underlying storage driver, since it operates on a block device abstraction.
Of course, some kernel modules have dependencies on other kernel modules, but those dependencies generally don't cross subsystem boundaries.
(There are of course exceptions to the examples I listed, but the argument should apply to the majority of device drivers.)
There's a phy driver for USB, phy driver for Type-C phy/mux, display port CDN driver, type-c TCPM controller driver, generic alternate mode DP driver, charger power supply driver, and more, and all these have to communicate together (and do so over extcon and type-c mux/role switch/orientation swtich interfaces)
USB 2.0 phy driver detects what kind of device is connected using BC1.2 specification (what kind of charger basically), TCPM driver does the same over USB-PD, they race together basically trying to identify how much current the device will be able to draw from the charger.
Because they are independent on HW and driver level, one of them takes it upon itself to get the information from the other, and decide what values will get more priority, and passes that over another interface to a charger driver, to make it configure the HW to not overload the power supply connected to the Type C port. This not a driver->subsystem but basically a driver<->driver interaction.
There's a ton more communication going on between these 7 or so drivers. TCPM figures out the peer device supports Alt-display port mode, so it will tell the type-c phy to reconfigure itself for 2 SS lanes and 2 alt-dp lanes modes to support parallel use of USB and display output. Displayport CDN driver orchestrates this, but there's a problem that link training is needed for this to work well, and that needs coordination between CDN-DP and Type-C PHY drivers, because there's no standard kernel interface for it.
So, yes I see a ton of inter-driver interaction. Many of it kinda ad-hoc, over some not perfectly well matching bus like interface like extcon or direct calls to other drivers, in addition to more common interfaces at the surface of various subsystems like clock, regulator, i2c, etc.
Maybe in x86 land this is saner, but in ARM SoC land, the drivers are often not that independent. Things need to happen across drivers in right order with right timing.
Not sure if that prevents these drivers to be implemented in rust, likely not, but the api surface is not small or simple, which was what I was trying to point out.
When they're running the DEC-provided X11 server, the mouse pointer moving on screen is done entirely within the graphics subsystem, with no need to talk to the kernel. Which leads to the hilarity of wiggling the mouse actually not being a good diagnostic for "has the kernel hung itself", as the mouse pointer will wiggle (but clicking doing nothing, and usually not changing shape as you'd expect as it goes over fields where you'd expect that).
I'm not sure I agree with this. As someone who does development for prehistoric PPC embedded systems, I get to take advantage of a lot of drivers that were never really intended for my platform, yet work just fine because they're based on standard busses like PCI and USB.
Consider some architecture supported by Linux, but not yet Rust. Now, consider that a vendor might create a driver for their fancy new PCIe network card in Rust with the logic being "well, this card was never intended to be used in anything other than an ARM or x86 system". There's a good chance that if that driver was written in C, it would have worked just fine (albeit not officially supported by the vendor) on that old non-Rust-supported architecture, but now that's not an option.
Rust should not be used in the Linux kernel until it supports 100% of the architectures supported by Linux itself.
https://doc.rust-lang.org/nightly/rustc/platform-support.htm...
vs.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... or https://en.wikipedia.org/wiki/List_of_Linux-supported_comput...
For instance, Tier 3 support for m68k-unknown-linux-gnu was recently stabilized in rustc as a result of m68k support landing in LLVM (https://github.com/rust-lang/rust/blob/master/RELEASES.md#co...), which has been one of the previous sticking points for Rust packaging on Linux distros, some of which still support m68k architectures.
Also, this is not going to retroactively affect any drivers, only the ones (potentially) developed from now on. As time passes, Rust support is expected to increase.
I don't think that's realistic.
If we buy the idea that Rust will allow kernel developers to write more robust, safer, more secure drivers, I think that should override a desire to allow things like driver portability to arches that hardware manufacturers never expected or intended. I would judge that support to be a "nice to have", but it shouldn't block what I consider to be more important concerns (security and robustness).
Regardless, the Rust support here is still considered experimental. I would be surprised if any company adopted it for a Linux driver for at least a year or more. That's plenty of time for people to get esoteric platforms supported in LLVM/Rust if they want to. And it also gives more time for the Rust GCC backend to become more mature and usable, and I believe that will give Rust support for every arch that GCC supports (more or less), which should cover your objection.
(At any rate, LLVM+Rust does currently support PPC, so your specific argument here isn't relevant.)
So what happens to users that depend on these archs & drivers - get left behind on the last non-Rust kernel?
edit: ok so it sounds a bit mean if you interpret it as if I'm not considering UnkleJoe to be a real human being. What I mean is, I don't think UnkleJoe is trying to get new devices to talk to his prehistoric PPC embedded system just for fun. An organisation is maintaining these systems, and that organisation is mooching off of Linux's excellent backwards compatibility. What's the economical cost of supporting that organisation, versus the economical costs of not implementing ksmbd in Rust?
Security updates for one. People are always complaining about insecure IoT devices/routers that get conscripted into botnets. Dropping kernel support entirely compounds to manufacturer firmware negligence - meaning end users can't update to the latest Lineage/DD-WRT/OpenWRT firmware, even if they want to.
> Most software runs fine on 10 year old linux kernels.
If you had to wrangle with world of custom Android ROMs or open source router firmware - even as a user and not a developer - you know that software does not work fine on old kernels. Trying to support both old and new kernels is a royal pain, especially if you have add shims to support old kernels that lack new features.
I thank $DIETY for Linus' pragmatic approach, I trust him to balance user support against developer zealotry. I wonder how long it'll take for Linux to go to shit after he retires. Kernel notes for v42.3: dropped support for Risc-V. Remaining supported archs are x64 and ARMv21-24
I thought we were talking about new drivers being developed in Rust. Your IoT security upgrade probably doesn't include a hardware upgrade, so it will not require these new drivers.
Regardless, it is the responsibility of the device manufacturer to support their old devices. It is their problem, hold them accountable.
The discussion moved to what happens to drivers that are not ported to Rust, and GP was arguing they should be abandoned/dropped - as opposed to requiring that all driver's be switched over to Rust first.
> It is their problem, hold them accountable
How can they be held accountable, pragmatically? Also, owning a CVE-riddled, insecure device is actually the user's problem. Open firmware can at least ameliorate the situation right now: where manufacturers cant be bothered to publish/upstream/update their drivers to the latest kernel API in the same language. Asking them to rewrite drivers in Rust and upstream is obviously a tall order - barring the passing of new legislation.
Why is it that so many people feel like they need to make sweeping pronouncements on the exact compatibility guarantees that a project as large as the Linux kernel must maintain across its entire code base?
Different maintainers of different subsystems of the Linux kernel make their own decisions on compatibility and support effort. There are a few basic principles shared across the project as a whole, such as user-space ABI compat guarantees, lack of kernel-space ABI guarantees, module licensing requirements, etc.
Not every piece of hardware even exists in the form of a PCIe expansion card. Some drivers are for peripherals that are embedded into a specific line of systems-on-a-chip. If I'm writing a driver for some peripheral that only exists on an AWS Graviton chip, I don't see any reason why some HN commenter should dictate that that driver can't be written in Rust just because there isn't yet upstream LLVM support for DEC Alpha chips.
I feel like most of these takes are due to someone getting upset a couple of years ago because some core library in Gnome like librsvg added a dependency on a Rust compiler back when rustc didn't yet support m68k systems, and this got someone working on packaging upset due to conflicting distro packaging policies.
The world moves on, however. The Linux kernel drops support for old hardware all the time. It introduces features that might only work on certain hardware. Also, rustc has added support for more platforms, including moving some platforms like aarch64 to Tier 1 support, and adding support for m68k.
While maintaining backwards compatibility or wide portability of drivers is a good thing, so is the much easier ability to write maintainable and secure code that Rust brings, so the desire to simply block one good thing that works on the vast majority of machines running Linux in favor of some small niche use cases doesn't really seem like the soundest decision.
If you need to support plugging in new hardware to a very niche embedded architecture, you will still have many options. These range the gamut from writing a driver based on the reference driver for your environment, to requesting/funding the development of such C driver from the vendor, to funding support for your hardware architecture in Rust/LLVM. Yes, all of these cost time/money. Maintenance of software ports to various architectures is very much not zero-cost, and assuming it is will eventually bite you or your business hard.
As another thought experiment: let's say the development of drivers in Rust is less expensive (either in $$$ spent on engineer time, in $$$ lost to bugs and security issues, or any other valuation you choose. If you/your community of people who desire new hardware to be supported by your embedded architecture can not cover this difference in cost (to make it worthwhile for drivers to be written for you), nor can afford to develop and maintain support for your architecture in Rust (so that you can benefit from the development work done by the vendor), nor can afford to upgrade to a newer hardware platform (which solves the problem completely), then your business is already screwed and you just don't realize it yet.
That said, this is the responsible way to introduce Rust code to the kernel - if this had be done by limiting future releases of Linux to only those supported by Rust, I'd be out with my pitchfork too, because that's something that really needs to be planned very carefully. As long as stable kernel releases continue to exist that support the full set of architectures, everyone's existing code will keep working, receive security fixes, etc. This is fine.
I'm joking of course, but the platform support list of rust is pretty good looking at [1].
I'm pretty sure people developing for arm5 aren't building the linux kernel on arm5.
[1] https://doc.rust-lang.org/nightly/rustc/platform-support.htm...
Linux has always favored support for currently (and in the foreseeable future) relevant architectures over widest possible spread.
I don't quite understand the desire to run recent software on museums piece hardware, but for those who must, there's NetBSD.
https://lwn.net/Articles/871283/ https://github.com/Rust-GCC/gccrs https://github.com/antoyo/rustc_codegen_gcc
I hope vendors would be more supportive of third party drivers because that obviously makes their hardware more attractive, but that has not been the case the last years.
I also don't have an opinion on the rust foundation. Member setup doesn't convince me, hope it doesn't influence the language negatively.
"This does not mean we will make it into mainline, of course, but it is a nice step to make things as smooth as possible."
https://www.phoronix.com/scan.php?page=news_item&px=Rust-Hit...
In my gentoo days I remember the compile time for firefox suddenly going up from 20 minutes to several hours which left a sour taste in my mouth with regards to rust.
Has there been much improvement in compile times in the past ~2 years?
Additionally, I would expect that the kernel would mostly avoid things that drive up compile time - generics and macros.
And as someone else mentioned, things should be able to be compiled in parallel.
Also, some crates for firefox are absurdly huge in terms of compile times - they are definitely outliers (IIUC). (Edit: I was thinking of this issue in Servo, so not definitely firefox releated: https://github.com/servo/servo/issues/1799)
* time to compile the syn itself
* time to run syn on the inputs
* time to compile whatever syn generated
To get a feeling how costly a simple proc macro is, run this benchmark: https://github.com/matklad/xshell/blob/4e5090e9f79baeed1037b....
> Building it takes almost 40 seconds
There's a lot of unavoidable generic types like Option, Result, RefCell etc. Do these make up any significant part of the issue with compile times?
https://arewefastyet.pages.dev/ - This page tracks compile times across some common crates over all supported compiler versions, with different hardware (2, 4, 8, 16 cores). This used to be https://arewefastyet.rs but the domain expired.
The tldr - speed ups of between 25-30% in these crates over the last 3 years.
One interesting observation is that helloworld now takes about 4 times longer[0] (though only to 0.8 seconds). What could have caused this?
Also, it’s possible that the last few releases might have sped it up a bit. That page only tracks till 1.52. I believe the latest release is 1.57.
Yes. Source: https://nnethercote.github.io/2021/11/12/the-rust-compiler-h...
In my experience, compile times are more a function of project structure than anything else. The compilation unit is a crate, so keep your crates nice and small, and you'll be fine.
So You Want to Rust the Linux Kernel? - https://news.ycombinator.com/item?id=28826246 - Oct 2021 (324 comments)
Rust for Linux redux - https://news.ycombinator.com/item?id=27939498 - July 2021 (204 comments)
Linux Rust Support - https://news.ycombinator.com/item?id=27746130 - July 2021 (433 comments)
Linus Torvalds on Rust support in kernel - https://news.ycombinator.com/item?id=26831841 - April 2021 (290 comments)
An RFC that adds support for Rust to the Linux kernel - https://news.ycombinator.com/item?id=26812047 - April 2021 (261 comments)
Linus Torvalds on where Rust will fit into Linux - https://news.ycombinator.com/item?id=26556459 - March 2021 (119 comments)
Supporting Linux kernel development in Rust - https://news.ycombinator.com/item?id=24334731 - Aug 2020 (354 comments)
Linux kernel in-tree Rust support - https://news.ycombinator.com/item?id=23800201 - July 2020 (491 comments)
https://lore.kernel.org/lkml/20211206140313.5653-1-ojeda@ker...
(it's maintained by kernel.org and is more official that lkml.org, which helped us a lot before we had an official solution)
The one you linked doesn't support the markdown in the post and it's hard to see the changes since it's in like 20 different emails?s.
* Compiling via rustc, mrustc, gccrs, rustccggcc
* Outputting MIR, macro expansion stages
If you have any idea of feature, don't hesitate to open an issue ! :D
Now that's just taking the piss.
This is untrue. Where did you get it? Have a look at kernel scheduler written in C++ here: https://fuchsia.googlesource.com/fuchsia/+/refs/heads/main/z.... constexpr! namespace!
If you look at the old code [1], it has a C++ extension & is likely compiled with a C++ compiler but the code itself is really just C. That's why cloc sometimes isn't an accurate tool for these. I don't have a sense of what the policy is but they could still be sticking with the C convention w/ C++ only utilizing things like constexpr & various other compiler pieces. They do seem to be making use of templates so it looks like there's been a lot more modernization done there.
[1] https://fuchsia.googlesource.com/fuchsia/+/9c7855e320662ad90...
Google has extremely strong internal coding standards. Those standards restrict which C++ features can be used, and how and where those C++ features can be used.
Without compliance based upon paycheck and performance reviews, I'm not sure that I'd want to try to make that kind of selective enforcement work.
-------------------------------------------------------
Language files blank comment code
-------------------------------------------------------
Rust 11813 339160 541958 2661720
C++ 9207 344592 198973 1632916
Go 2906 81163 138138 608871
C/C++ Header 7544 156294 209786 559658
C 1680 49614 52841 260405
Assembly 322 18112 11620 133536
-------------------------------------------------------
Header + C + Assembly 9546 224020 274247 953599If anyone's interested - here's Linus on C++ in the linux kernel: http://harmful.cat-v.org/software/c++/linus
Windows has been slowly migrating to C++ since Vista, when VC++ got kernel support for C++ code, naturally one just doesn't move something like Windows to C++ just like that.
https://community.osr.com/discussion/291326/the-new-wil-libr...
> First off, let me point out that this library is used to implement large parts of the OS. There are hundreds of developers here who use it. So unlike, uh, some other things that get tossed onto github, this project is not likely to wither and die tomorrow.
Also, both windows and mac has considerable and growing amount of C++.
Certainly the /more/ low level you go, the less C++-isms you see.
These days, sure you can use a highly restricted form of C++ in systems close to HW, but you might as well be using the industry standard in those environments, which is C, compiled with a C/C++ compiler.
You'd be surprised how many fortune 500 software companies have thousands of programmers who are writing new C code every day.
Even Apple writes new C every day.
C++ is also an evolving monster that everyone uses different subsets of, and many developers only know or use subsets of, either for lack of expertise or out of conviction (see how polarised the discussion of exceptions in C++ is here).
And every five years, even more complexity gets added, making the C++ story a modern PL/1-like story.
For many decades, there were also substantial differences in compiler maturity, forcing many to rely on subsets artificially.
At least C++ language features will have proper documentation, not this “if we place a struct inside this one at a specific place, we can use it as a linked list for no added memory cost” shit that one must know for each and every project (and is error prone).
The only major one I know of was some push for C++ which Linus wasn't a fan of: http://harmful.cat-v.org/software/c++/linus
One important factor to consider there, and for which Linux is wholly unlike the vast majority of other projects, even system projects, is that the userbase is just massive, and includes a lot of the world's infrastructure. That results in a rather different cost/benefit calculus around abstraction/performance tradeoffs than the ones for which object-oriented and functional programming were designed.
For example, I think that it's probably good for all of us that there's not really a whole lot of dynamic dispatch happening in the Linux kernel. I would generally prefer that my syscalls keep pointer chasing to an absolute minimum. But I'm hard pressed to imagine how a C++ style that avoids vtables like the plague could really have that much to offer over straight C. And it would simultaneously put extra code review pressure on the kernel team, since they would have to carefully watch to make sure contributors aren't introducing such things.
Similar concerns apply to Rust, mind. But it's much easier for me to see how Rust without trait objects could still bring something of value to the table.
C++ would be a huge win all over the kernel, the only barrier to it is Linus' ignorance.
unique_ptr / shared_ptr alone would be a massive upgrade over straight C. Simply being able to enforce ownership would be huge. So would just having namespaces. Things like move semantics, static_assert, and constexprs would then be the cherry on top. There's also public/private fields, friend classes, etc... to keep people from accidentally poking things they shouldn't be.
C++ is so very much not just "C with vtables."
But C++ has a reputation, deserved or not, for being overly complex. And ultimately Linus chose the devil he knew over something that by all accounts would have been better. Nearly every other large C project has moved into C++, even from other staunch defenders of C like John Carmack eventually relented that C++ was better for large teams.
Rust gets to hit the reset button here with a fresh reputation (and some extra safety & cool tricks), and Linus' views may have changed over time.
Examples include exceptions as the one true error mechanism, and the use of implicit allocation.
Rust doesn't have the former, and Rust's core doesn't have the latter (but std does), one of the stipulations from Linus when this began was that Rust for Linux needed to ensure it didn't sneak in implicit allocation. Thus, while a beginner's Rust program might have text += "Some more stuff"; you won't see that in the kernel -- because it won't compile, the core::ops::AddAssign isn't implemented on these string types in the kernel because it's implicit allocation.
As to move, the move semantics in C++ are really awkward. They're a nice optimisation of course, which is why they're provided, but you would not deliberately do it that way in a language which had move semantics from the outset, as demonstrated by Rust. To make move work with existing C++ software it has to "hollow out" objects when they're moved from. Rust needn't do that, if you move a Foozle from var1 to var2 in C++, it must carefully hollow out the Foozle in var1, because in C++ var1 will eventually get destroyed, so its contents must be valid and safe to destroy. If you move the same Foozle from var1 to var2 in Rust, the Rust compiler knows you moved it, so, it's gone, var1 won't get destroyed and no further attention to it is needed.
-fno-exceptions is very widely used and non-throwing variants of basically everything exist as a result. That's not a reason to avoid C++ in the kernel as a result. Especially with how heavily Linux already relies on GCC C extensions, there's obviously no hangups on being "strictly standard compliant." Not have exceptions ever been C++'s "one true error" mechanism ever. For example see iostream: https://www.cplusplus.com/reference/ios/ios/rdstate/
C++ is not, and never has been, anywhere near that dogmatic.
As for the rest of your points, you seem to be arguing Rust is better than C++. But that's not really relevant to why C++ was rejected before Rust even existed. I do think Rust is even better than C++, but I also think C++ is a huge improvement over C, especially for large codebases.
However whereas you can write -fno-exceptions and get a C++ with this core language feature excised, you can't do the same for implicit allocation.
In contrast the Rust for Linux team have worked hard to produce a Rust for Linux environment that doesn't panic!() and has no implicit allocation, since these are red lines for Linus.
You're still trying to force C++ to be dogmatic. It never has been. It's always tried to be accommodating to all types of usages. It's always been multiparadigm. 'new' wasn't even defined to throw std::bad_alloc until C++98, which also introduced 'std::nothrow_t' to have it return null instead.
> you can't do the same for implicit allocation
Just like with exceptions this really isn't true, either. There's no implicit allocations in the core language, only in the STL. Saying that the "new" operator does implicit allocation is like saying "malloc" does implicit allocation. Regardless, placement new has always existed, and the non-placement 'new' operator has always been overridable. And all of the STL is optional & replaceable.
C++ has become lots of little silos of people using dialects that are in practice mutually unintelligible with other C++ dialects. Hence as I said the concern of the committee about this problem. What's the purpose of a "standard" which doesn't standardise anything?
But to be sure, Linus could create yet another silo.
And for implicit allocation, like I said, Rust for Linux did the work. C++ practitioners didn't. Linus isn't going to take something that doesn't exist just because its fans say if it did exist it would be suitable.
I don't follow this argument. In my experience, use of vtables is relatively rare in modern C++ development even when you aren't trying to avoid them, many of which are elided by the compiler anyway. Regardless, in these infrequent cases it is straightforward to code equivalent functionality without vtables, since it is syntactic type-safe sugar over the code you'd have to write in C anyway. They are like C bitfields in that every systems programmer knows how to implement the equivalent functionality manually for cases when the language feature is underspecified for the purpose.
Modern C++ without vtables is only marginally different than modern C++ with vtables, the code you'd write and the features you'd use would be nearly unchanged.
Between this and the other comment where you say Linus dislikes OOP I kinda get the feeling you ought to read some Linux kernel code. Because it's practically completely object-oriented and almost everything is done through a vtable for dynamic dispatch.
C++ has moved on from the days when that argument was made. This may have sufficed for an argument against C++ then but it should also apply to rust today which has similar infrastructure like generics and traits that allows a level of abstract programming that C doesn't provide.
I think we ought to use the new MAMAA.
C#, Java? Managed code with JITs, not supported on a kernel level no-way no-how. Ditto for Python, PHP, JavaScript, and other high-level languages without pointers and all that jazz.
Go would be close… but it requires bundling a bunch of libraries in with each binary and wasn’t designed for low-level development. Again unsuitable.
C++? Linus Torvalds and the kernel developers hate it for being unnecessarily complicated and only making things harder to maintain while bringing little to the table at this point (they’ve made it this far without classes, and C++ doesn’t bring anything new like memory safety).
D I think suffered from a proprietary compiler and wide use of GC. But I’m not really familiar with it.
Of course it doesn't make much sense to use it in the Linux kernel but it is a low level (or low level capable) programming language that can be used for systems programming.
The gap between Pascal with a couple extensions and C, or between Object Pascal and Objective-C, is very small, anyway. Most semantic elements have a one-to-one correspondence. There were even UNIX clones written in Pascal.
No it is not. There is Ada/SPARK and has been for a long time, and it is for critical systems. I recommend checking out https://blog.adacore.com/ for examples, there is plenty of that. For one: "An Embedded USB Device stack in Ada". There are also many articles about SPARK. For example: "Enhancing the Security of a TCP Stack with SPARK". These are just one of the recent articles.
People seem to love Rust. I've never seen that kind of enthusiasm for Ada.
Are there any non-proprietary implementations of Ada/SPARK which don't place limitations on the compiler output?
[0] https://news.ycombinator.com/item?id=24492653
[1] https://old.reddit.com/r/ada/comments/3989f4/gnat_gpl_restri...
For better or worse, the ship has probably sailed on Ada/SPARK making its way into mainline Linux.
Not for Linux, although they work perfectly fine in industrial installations running bare metal JVMs.
Exceptions are necessary to enforce class invariants.
The same argument could be made with return values: what happens if you don't handle them and just panic on them.
Having exception-worthy invariants become panics would probably be a nightmare, at least for idiomatic C++: every nontrivial ctor (including copies and moves) might be a panic site.
While there are arguments over when and how to use exceptions, I think this is the first time I've seen them positioned as a killer feature of C++.
It's not a killer feature of C++, it's a requirement, without exceptions you can't have faillible ctors meaning your object lifecycle suddenly becomes a lot more complicated and full of additional traps.
I covered this option:
> your object lifecycle suddenly becomes a lot more complicated and full of additional traps.
because without ctors you don't have object invariants as you have to split instantiation and initialisation, or you have to implement bespoke and unusual strategies drawing you further and further away from anything which could be described as "standard" C++, thus making the advantages of C++ dwindle further into the distance. And all that with, as far as I know, no standard tooling to support you.
Classe invariants must be kept in each method call, not only constructors and destructors, and again they only matter in classes.
C++ is a multi-paradigm language, this is not Python Zen.
As a related but separate thought, sometimes adding invalid states to an object is useful. For example, take a look at std::thread::detach. Sometimes you see rvalue qualifiers on such methods (indicating the object has been consumed).
If you consider memory allocation to be infallible, there is a lot of stuff that can be written as pretty normal c++ without exceptions. The STL even pretty much works without throwing given this. If you consider allocation to be fallible you're going to have a bad time with "standard" c++ too.
And detaching a thread is pretty much always a bad idea.
The problem is not the ability to do it but the ability to enforce it. "Just be careful" has not worked, and will keep not working. Google's coding standard has not stopped C++ from causing them problems.
> As a related but separate thought, sometimes adding invalid states to an object is useful. For example, take a look at std::thread::detach. Sometimes you see rvalue qualifiers on such methods (indicating the object has been consumed).
You know what's actually useful and good?
For the object to actually be consumed and not be accessible at all anymore.
> If you consider memory allocation to be infallible
Which you can't in the kernel.
> If you consider allocation to be fallible you're going to have a bad time with "standard" c++ too.
Yes? That would be the point.
Disabling exceptions gives you C with classes, which is the old style.
I write ultra low-latency system-level code for HFT, and I certainly use exceptions.
Clearly, modern C++ has very little to do with exceptions, or I would have required them or found them useful at some point. I’m not saying exceptions are not useful, but their centrality to the language is empirically false given the vast number of modern C++ code bases that don’t use them and lose nothing by it.
Also when speaking about kernels, it isn't as if the other languages being discussed (including C), make use of their respective stdlib in kernel space.
If your process doesn’t survive allocation in bootstrap then it is badly misconfigured. If it survives bootstrap, then the software has been able to make hard guarantees about its resource constraints; the runtime will optimize to those constraints. Many years ago, given the designs of the time, you had to worry about blowing out the stack memory occasionally in edge cases, but even that isn’t a real problem these days due to changes in how it is done.
You would design things this way in any language. The benefit from a software design standpoint is that you can derive an enormous number of invariants from these constraints that can be used to guarantee that the core processes will never experience resource exhaustion. This eliminates a massive number of edge cases that would have to be accounted for if resource exhaustion was detected by testing limits.
C++ also does not have decent macro system, which is leveraged in Rust error handling (the derive system).
Yeah macro like tooling in C++ is clunky, but on the other hand I think Rust goes overboard with macros.
C# has pointers and can be compiled to native code. I think the main problem is GC which is required for most of the standard library (you can avoid heap allocations but that would be unidiomatic)
AFAIK until recently (with the introduction of the Span<T> types), the only way to avoid heap allocations for arrays or any other kind of collections in C# was with stackalloc, which is unsafe.
Rust is actually heavily influenced by C++, but enforces that those practices are respected as part of the language.
Where I'm less convinced that it's an option with a high likelihood of success is in open source. It's one thing when you've got a $250,000 salary riding on you ability to scrupulously follow best practices. But results may vary when the work is being done on an amateur basis.
Sounds like a great way to have a toxic workplace that either yells at people for bugs, or never gets anything done because people are paranoid about committing bugs (or both!). Meanwhile, with a different language you get the compiler yelling at you instead. Automated enforcement uber alles.
I don't think open source vs commercial dev really matters that much.
Much, if not most, of Linux kernel development is done by employees of various companies, who are being paid for their work. Typically, that's in the form of device drivers, which also happens to be where I've seen the shoddiest, buggiest code in the kernel [1].
[1] Think "driver A abusing the 'private' pointer of driver B for its own purposes" kind of buggy.
In aerospace you probably want all this anyway, but for others, where 99.99% is enough, there needs to be a cheaper and more efficient way.
2. unsafe blocks are explicit and therefore visible during code review. Memory safety violations in C++ are not always visible, simply because they are implicit.
3. Thanks to (2), unsafe usage in a codebase can be easily quantified and, if needed, audited.
The harder problem is convincing people that raising their quality standards is a necessary requirement for them to contribute to the code.
Secondly, quality standards rely on the human element, which we know is fallible and never 100% reliable. Compiler-enforced memory safety checks are (in theory) infallible[1].
[1]: Infallible when all of your dependencies are also verifiably memory safe. In reality, this is not the case, but I believe that Rust can reliably get much closer to 100% than usual.
Doesn't this assume that there are no bugs in the Rust compiler (for "safe" code)? Isn't that still somewhat of a big assumption at this point? I get that is indeed a very core goal of Rust, but has this been proven for any/all released versions of the Rust compilers?
Long story short, for the vast majority of use-cases, developers can assume that the compiler and stdlib are trusted. This is also how Rust defines "safety", which leaves you to focus on your code and its dependencies.
But "guaranteed" that there can be no security holes in code marked "safe" due to an inadvertent compiler bug? I would be very surprised if that is actually the case.
Less likely than C/C++? Yes of course. Guaranteed? Probably not.
Will the Rust compiler have bugs in practice? Of course. Do you - as a developer - have a different safety model than the Rust language (i.e., one that does not include the compiler)? If the answer is no, you can “ignore” compiler bugs from a safety perspective.
Addendum: yes, that last line is what pisses a lot of people off with Rust, and honestly, rightfully so. I've written quite a bit of Rust by now, and getting your code to work can feel like a bit of an escape room. Having C as an option on the kernel alongside it is a great idea though, because there are situations where I'd happily pick up C over Rust, and they're not just 'edge cases' where Rust falls short. Ideally, they're two different tools that can be applied depending on the situation at hand.
Those are the most elementary rules that one must follow when writing C++.
Doing something correctly is not a best practice though. If it were, we could just say "if you correctly manage memory".
What happens when you mess up? What happens if you incorrectly model ownership with RAII? Or is it not really possible?
And what basic guarantees are not enforceable without ownership?
> With Rust, you actively have to fight the compiler to cause memory leaks.
Memory leaks are not considered unsafe in Rust[1]! They're actually an important part of Rust's lifetime semantics: intentionally leaking memory is both safe and equivalent to producing a `'static` out of thin air.
You probably shouldn't leak memory as a matter of practice, of course, but Rust will happily allow you to do so in 100% safe code.
When rust was proposed for linux, controlled memory allocation was on the top of requirements list
Another good example is implicit comparison conversions, e.g. some class defines `operator==` as non-explicit and any code that does `x == y` where `typeof(x) != typeof(y)` may find itself going though a possibly-allocating constructor.
But even beyond that: you can have a non-trivial constructor that requires heap allocations internally. So even relying on (sane) compiler-defined behavior won't save you in the general case.
Presumably the same is true of C.
But that aside, it just seems like a very weird choice that would have profound perf implications - to the point of removing pretty much the sole remaining advantage of the language - for no gain whatsoever, since it's not even simpler.
As for non-trivial ctors - at that point you might as well worry that every function call made in pure C code can also heap-allocate, no?
https://godbolt.org/z/re51ensMe
terminate called after throwing an instance of 'std::bad_optional_access'
what(): bad optional access
Go has a garbage collector and doesn't allow for manual memory management which would rule it out entirely. This adds a requirement for the go runtime, which is a further nonstarter.
I've notice that the trend of new compiled language is fairly recent with Nim (2006)/Go (2009)/Rust (2010). The previous trend was more oriented toward interpreted languages Python (1991)/Ruby (1995)/Javascript (1995).
So maybe, 10 years is the maturity age and we will see more of those language used for Linux kernel development.
Other candidates like Modula-2 or Ada, never had big sucess in UNIX space.
Objective-C never went beyond NeXT and Apple (yes NeXTSTEP drivers were written in Objective-C).
“I will not use C++ and programmers who develop projects instead of C are kicked out, lest they mess up the projects I’m involved in”, “C++ ended up with a bunch of terrible, unmaintainable garbage”
So how could C++ ever be possible in the Linux kernel?
The problem isn't technical, and to come back to your C++ example, there are plenty of OSes using C++ at the kernel level.
macOS being one of them via IO Kit, Windows since Vista, or that maker board loved by everyone, named Arduino.
Probably to Linus dismay, all Project Treble drivers (which run in userspace) on Android, are mix of C++ and Java, classical Linux drivers are considered "legacy" in what concerns AOSP project.
The only other language that could of had a chance of derailing the use of C on Unix and related was Pascal, because it was quite popular in many universities at the time and of similar capability. Consequently, people working for AT&T or connected to C development, viscously attacked Pascal in various papers and there was a lot of bad mouthing in the American media to help make sure it would not become such a threat. With AT&T so strongly pushing C and Unix, they got the result that they wanted.
Niklaus Wirth, the creator of Pascal, didn't defend his creation because he was preoccupied with Modula-2 and later Oberon. Even so, he was just a university professor, and any defense that he might have tried would have likely failed anyway in the face of AT&T money and connections.
There might be objectively better languages for kernel development out there, but it doesn't do any good if nobody uses or knows them.
Others, like Go, use garbage collection. That may not be universally unacceptable in OS kernels, but it's certainly not something anyone wants to see happening in the Linux kernel, which sees use in applications with low-memory and real-time constraints.
And the rest are dead or dying languages such as Pascal and Ada.
I can't speak to Rust's merits, I don't know it well enough, but if some reasonable subset of Rust makes it easy to stick to C-compatible ABIs without sacrificing most of the language's practical differences from C, then I would say that, yeah, it may truly be the first good - or even reasonable - candidate.
The biggest reason why Rust is, in my mind, a contender for the kernel is that it genuinely adds a new capability that C++ does not: true and guaranteed memory safety, with an unsafe core that is much easier to audit. Drivers written in Rust will be much less likely to contain memory safety problems, and that’s potentially a big win for kernel maintainers and developers. And yeah, it’s also interoperable with C to a large degree, which certainly helps.
Kind of a memory safety equivalent to what structured programming did for control flow.
I don't quite get this odd behavior of so quickly labeling a language as "dead" and devoid of any evidence to back up the claim. In the case of Pascal and Ada, various commercial interests want them to be "dead", because both represent a threat to their financial interests.
Pascal became Object Pascal, which is very much around in the form of Delphi/Embarcadero, Free Pascal/Lazarus, Oxygene/Rem Objects, PascalABC, etc...
Ada for sure is not going anywhere for many years to come because of its use in the military and aerospace industries.
- C++ - Rust - Zig - Ada
Rust has many qualities, but it's not a simple language.
The problem with complex languages is that they are ill suited for low level stuff.
Sometimes it's better to not have high level language features, because it allows everybody to understand the code without having to know every feature of the language.
Simple is better. There are too many developers who prefer complexity.
Certain problems have an inherent irreducible complexity. This complexity has to go somewhere. If the language doesn't tackle it, then users of the language will have to.
This is especially clear in the case of memory errors in C. C is too basic to express ownership of memory, valid scopes of pointers, and thread-safety of their usage. This doesn't make these problems disappear, only requires users to handle it manually and get it right without compiler's help.
In Rust you have to specify ownership and thread-safety of everything. It is complex, but not because Rust is complex, but because safe low-level memory management and thread safety are difficult problems.
Problem I have with Rust is that it uses certain patterns to make it work against the unforgiving borrow checker. I am not convinced that we will stay at this point.
Rust disallows some operations where the developer can know the operation to be safe, but the borrow checker does not. So you need to accommodate the language.
It's not a blocker in practice, because you can fudge things with `unsafe` if you know you're smarter than the borrow checker.
This is going to be unpopular opinion because it is more philosophical and completely unscientific.
I have been thinking something along that line for quite some time and not just on technology or programming languages. I think it has a correlation with age or how far along your journey in life. Simplicity has to click, it cant be told or learn just by reading. And more often that not, you can only understand simplicity after you try and fail to "fully" embrace complexity.
If you want a truly simple language you should have a look at Go.
Sure it's simple in a sense there are not many keywords or features. And you can learn it over a weekend and start programming in it. But when you factor in undefined behavior, straight up user hostile choices done by common compilers in the name of performance, common patterns/library functions (that are all over books and stackowerflows) to avoid etc. C is not that simple anymore.
When you throw in concurrency, programming in C is quite hard.
I write this because I used to be an embedded C programmer, but that was over 10 years ago. I can still read code, sometimes even add simple patches (not to kernel ), so you could say I know the language. But it would take me several months of focused training before I would even dare to touch something as linux kernel.
Not saying that Rust is simple, just saying C isn't as simple as some people make it.
Also, all of these are low level languages. There is no real difference between C and C++ in that way. C just lacks expressive power compared to C++.
It is the "first language" that Linus didn't object to. He hated both C++ and Ada. I am not entirely sure if he really likes rust, sees borrow checker as something important worth trying, or lost to the pressure from RESF ( Rust Evangelism Strike Force ) within the linux kernel dev community.
He is a proponent of C because he can imagine, or envision the generated assembly from C compiler. Something like a High Level of assembly. And something along the line of people who can write good C code tends to have all the scar tissue to prove it.
>What do you think of the projects currently underway to develop OS kernels in languages like Rust (touted for having built-in safeties that C does not)?
>That's not a new phenomenon at all. We've had the system people who used Modula-2 or Ada, and I have to say Rust looks a lot better than either of those two disasters.
>I'm not convinced about Rust for an OS kernel (there's a lot more to system programming than the kernel, though), but at the same time there is no question that C has a lot of limitations.
And I asked him once I think in a private email a few years back and he manage to gave me a long detailed reply that was like an essay ( Still very very grateful for his time ). May be he doesn't hate Ada as he did to C++, but it certainly feels he doesn't like it.
I would be interested to see his opinion on Zig, which I think really fits his liking. But may be after 1.0, which is still some way off.
I think Linus also commented that it has to be exciting and interesting for developers besides being useful obviously.
People are talking about Linus' opinions of some other languages, but more importantly the Rust folks have been prepared to put in a lot of work to meet Linus' concerns. You're now seeing all sorts of other language advocates pop up with D, Zig, and so on, but they've not put in the same work to convince Linus that anything else are real options.
First, a bit of background: I am mainly a C++ and C# user space developer, so presumably despised by all camps involved. Secondly, I have no experience of kernel development whatsoever. Thus, take my comments with a large grain of salt.
My argument is as follows. Every decision you make on a software project is akin to an intersection on sets of programmers. When you choose C, you take a small slice on all possible programmers; when you choose kernel development, you take a slice on the set of competent C programmers, and so on. The intersection of programmers that are competent in Rust _and_ in C _and_ in kernel development - including the interaction between the two languages, at a very low-level (because kernel-space is special) - must be astonishingly small. And herein lies an important problem. Let us posit that the kernel does gain a significant amount of code in Rust; the interaction between these two languages (read: friction) will become significant. Either the Rust people will break things for the C people, or the C people will break things for the Rust people. Those who like C but not Rust - presumably a large subset of the Linux C programmers, else one would assume they would be developing a kernel in Rust - will become alienated. Similarly, those who like Rust will be forced to spend a lot of time doing non-rusty things just to get Rust to work, and they too will not enjoy the experience. Finally, many of the sweeping code clean-ups the kernel experiences periodically will apply differently to Rust, if they apply at all - meaning it will be a second class citizen, barring some major investment to compensate for this.
In the end, I do not think the problems will come from the languages involved per se but due to the complex technical and especially social interactions that this will create.
It is, however, an extremely interesting project technically and I suspect that Rust (and maybe even the kernel) will benefit from it, though not necessarily in ways one would expect. As far as research goes, it is pretty cool.
Types more advanced than Vec<T> should be avoided IMHO. Rust is a beautiful language when you can contain it's complexity to a minimum. Them it reads like safe C, just like it should.
Naturally, they have fallen into the same trap C++ did; they failed to make the language easy to read and write by introducing too much complexity.
Fortunately for us. Though calling it "better C++" is underselling Rust by quite a bit IMHO.
Why?
1. "Don't put ifdef"
2. "Why?"
3. "Put this other ifdef"???
I'm sure everybody reading HN knows who Google are, but ISRG are the not-for-profit behind Let's Encrypt among other things:
I'd also be very interested to hear a Rust for Linux perspective on the current state of async Rust. I've been avoiding diving into it for years now, but I'm always looking to be convinced I should.
In 2021 I've had 145 rust-panic involving firefox crashes.
Solve what issue?
If you're talking about the problem of, "Linux is available on more systems than the Rust compiler has been ported to," then I'm gonna say "no," because many (if not all) of the core issues that make Rust difficult to support well on various architectures (and hence why said support isn't in official Rust today) would also have to be solved by any so-called "Rust-to-C transpiler."
Alternatively, LLVM byte code to C compiler would do the job, as rustc emits LLVM byte code afaik
As awful as C is, there is real value in keeping kernel development limited to one language. If people really want a Rust kernel, try Redox
You need to learn language concepts like '?', derives, traits, dyn traits, etc, but they're all fairly straightforward.
Rust's big barrier to entry is learning to write code that passes the borrow checker, but that's not a concern when doing code reviews.
Rust code is actually easier to review in that regard because (unless the code contains unsafe blocks) the reviewer can skip thinking about most aspects of memory management.
However, in reality Linus wont necessarily review the patch line by line. Mostly the reviews will happen on mailing lists. Then if Linus' lieutenants that handle the specific subsystem like what they see, they will probably give a general thumbs up for the change. Finally Linus will take an executive call. He may offer some technical suggestions (as he is technical) and send it back or accept the patch or conditionally accept the patch.
The point is that the finer details will have already been dealt with by the time it lands on Linus' desktop.
But given that this is going to affect a lot of things for years to come he is probably going to spend a lot more time on it than other features.
Also I think Linux may be interested in having Rust in the kernel. See here https://lkml.org/lkml/2021/4/14/1099 . He says "on the whole I don't hate it".
It's clear to see that there is a lot of momentum behind Rust. Allowing Rust in the Linux kernel will help bring a lot more talent and energy into the kernel community (other than the inherent merits of Rust which I think are easy to see).
It is needed and will be more cohesive and take more advantage of the new language than stuffing it into Linux.
There will be a lot to learn from it though. It will test the interop and cohabitation of Rust and C. Lessons learned will be unbelievably valuable for other efforts to mix and match them.
Why is it needed?
WindowsNT is by far the most modern.
Linux is a not terribly good copy of an operating system made to run on a PDP11. WindowsNT owes quite a bit of internal architecture from Vax.
Just like there is constant experimentation and development of new programming languages there should be constant experimentation and development of new operating systems.
Limits, metaphors, hardware, has changed to an extreme degree, there is no reason to stick with that made sense decades ago. :)
Where is the Tesla Operating System, (well in something a bit new and fresh with some good ideas and a lot of problems)
In my day job I do data engineering, and I'm a long-time linux user. Multimedia stuff aside, I think it's been 8 years since last time I felt limited by the operating system.
Granted, I don't do anything particularly low-level. But as a high-level "consumer", I experience the main issues with today's operating systems not to be technical capabilities, but rather user friendliness, flexibility, and standardised APIs.
But I'm curious what the issues are from a lower-level point of view!