Linus on Rust and the Kernel DMA Layer
lwn.net
lwn.net
But maintainers claim that other people simply interfacing with their code through Rust bindings would require the maintainers to change that Rust code anyway if any changes they make would break those bindings? Am I getting that right?
If those places now include Rust then it would require you to learn Rust (or depend on other people).
Really they just need to make a decision at this point. Either Rust is allowed and maintainers are going to have to learn it, or there's no point and the Rust people are going to have to fork Linux.
I guess there is a third option: a stable driver ABI. But they've avoided that at all costs for years so I don't see it happening. And besides, Rust is going to be used for more than just drivers eventually.
They already _have_ made this decision. Why do people keep misunderstanding this. While ordinarily if you change the API you are required to change all of the callers of that API to match, in this one special case the maintainer of a module can declare that they don’t want to touch the Rust code and that if they change the API then they are not going to change the Rust code that calls it. In that case the Rust for Linux team will step in and make whatever changes to the Rust code are needed.
I don't do mainline Linux dev regularly, I can't say whether the claim that Christoph made that the process doesn't actually work that way in practice is true or not without doing a lot of archaeology, but that's why there's confusion around it - because some people keep insisting that it doesn't work that way in reality. (At a 500 mile view, it seems like Linus going around making very explicit statements shutting down claims of that uncertainty a bunch would do wonders for clarity, but that assumes good faith on a lot of people's parts, and I also think Linus has learned not to be too visible in his proclamations about anything if he can avoid it...)
Did you read the message? This is Linus making a very explicit statement shutting down the claim that there is uncertainty.
My remark is that the complaint that kept coming up from both proponents and detractors of the R4L effort is that Linus said one thing in writing and the actual outcomes appear to be substantially different, and at the point where several people who have contributed for years to Linux have left over the friction involved, I don't think "he should have intervened sooner" is a novel take.
With this rule that the Rust developers must maintain their own code, they are in the same situation like the maintainers of out-of-kernel device drivers.
I have maintained out-of-kernel Linux device drivers, so I know that this is a huge PITA. Almost every new Linux kernel minor version was breaking the device drivers, due to interface changes like: moving definitions from one Linux kernel header to another, adding or deleting parameters to functions, adding or deleting members to structures.
It is not enough to say that the Rust developers must maintain their own code, but it is also necessary for the responsibilities of the maintainers of the kernel subsystems written in C to be changed. Whoever changes an interface should also have the obligation to write a concise migration guide in which to explain what must be done to convert the invocations of the old interfaces into invocations of the new interfaces.
Only in this case the Rust developers would be able to maintain their code. Otherwise, they would have to bombard the C developers with questions after each change, which I assume is what Christoph Hellwig fears.
Such a requirement will increase the workload for the C developers, so I am not surprised that they are reluctant to accept users who will maintain their own code.
I have not checked recently the kernel documentation, but I doubt that anything has changed. When I was maintaining device drivers, every time when they became broken the only thing that could be done was to scan the kernel E-mail lists, to find who and why has changed the interface. This provided some context about the purpose of the changes, but there has never been any information about how to migrate the existing code to use the new interfaces.
When function arguments or structure members were deleted, it was simpler, one could delete them also in the device drivers hoping that they will continue to work. That did not work always, because sometimes additional new interfaces had to be invoked, to compensate for the simplification of the old interfaces.
Much worse was when function arguments or structure members were added, because only seldom it was possible to guess what values should be put in them.
In many cases the only solution was to waste a lot of time with reading the source for the entire subsystem whose interface had changed, in order to understand which were the effects of the interface changes.
All this was simple, but very tedious. What could have been done in a few minutes, had a migration guide existed, could require many hours of work in the absence of adequate documentation.
Unless the requirements for the documentation of the internal interfaces change, this is exactly what the Rust developers will also have to do, with the additional difficulty that they may be not very proficient at reading the kernel C language sources.
If there would be a clear requirement for the C maintainers to write a migration guide whenever they change interfaces, that would help not only the maintainers of Rust Linux device drivers, but also the maintainers of any out-of-kernel Linux device drivers.
However, there have been some Linux developers who not only did not make easy the maintaining of out-of-kernel device drivers, but who have made efforts to make as difficult as possible the maintaining of out-of-kernel device drivers, apparently due to some misguided belief that this will hurt some evil corporations.
In my experience, most out-of-kernel device drivers are for obsolete devices or for devices with very few users, whose maintenance is not sponsored by any corporation. So this behavior of actively making difficult the maintenance of out-of-kernel device drivers hurts mostly hobbyists, or at most small businesses, because they do not have the resources to push device drivers into the official tree.
No, certainly not forever. In the short term I think it is a workable solution. I think that over time more and more maintainers will realize that maintaining Rust code is easier than maintaining C code, and that there is actually very little added burden to maintaining the Rust bindings and abstractions that wrap their C code. The Thus the Rust team will not have to do that job forever.
> Whoever changes an interface should also have the obligation to write a concise migration guide in which to explain what must be done to convert the invocations of the old interfaces into invocations of the new interfaces.
That is an excellent suggestion :)
The migration guide should be part of, or even the bulk of, a well–written commit message.
> However, there have been some Linux developers who not only did not make easy the maintaining of out-of-kernel device drivers, but who have made efforts to make as difficult as possible the maintaining of out-of-kernel device drivers, apparently due to some misguided belief that this will hurt some evil corporations.
And that doesn’t surprise me at all.
I would imagine the hopefully straightforward path would be to make sure the Rust code gets build tested, and then if that fails, have a route to reach out to the people who want to do R4L work and tell them "this is breaking because of what I changed, can you help when you get a chance", for the maintainers who don't want to learn Rust.
...of course, one reason the maintainers loathe this idea, is that the personality archetype who works on Linux generally is the kind of person who would go fix it themselves because they're impatient, so with that framing, it's pretty clear why they think "wait on other people to fix it" is a nonstarter de facto...
I understand that's standard if you know you broke them, I was imagining a solution to the problem "this is broken and needs to be fixed, preferably quickly, to unblock me, but I don't want to go down the Rust rabbit hole", since by nature, I would not expect most of the R4L maintainers to want to volunteer to do that kind of turnaround time on a consistent basis.
(This might be an unnecessary optimization if they're already planning for that, but speaking personally, I would think a lot of the fixes involved would probably be somewhat mechanical on the R4L side for that kind of breakage, and I also think it would provide something of a "happy path" for maintainers, in terms of perception, to know that they have an "in case of Rust do X" ramp even if in practice it rarely if ever gets exercised...)
How would Rust breakage block someone who only writes C?
> I would not expect most of the R4L maintainers to want to volunteer to do that kind of turnaround time on a consistent basis.
There's at least six people who are being paid full time to work on Rust for Linux, across a few different companies. I would imagine fixing any breakage is a high priority for them, and since it's part of their job, I'd imagine they'll get it done.
He NAKed one patch. One single patch. And the reasoning for that patch was that everyone was in the holidays. Otherwise he was willing to ship a broken rust kernel.
Others have pointed out that this is mistaken, but also, it's in the link itself:
> So when you change the C interfaces, the Rust people will have to deal with the fallout, and will have to fix the Rust bindings. That's kind of the promise here: there's that "wall of protection" around C developers that don't want to deal with Rust issues in the promise that they don't have to deal with Rust.
I wasn't mistaken; this is exactly what I thought. The problem is that Rust developers aren't magical genies who you can summon to fix Rust code. In the real world they are volunteers who might be busy, on holiday, no longer interested, dead, etc. What do C maintainers do when the Rust genies don't magically appear and fix their code? Just break the Rust code? That's obviously not a good long term solution.
So while I totally disagree with the C developers - they should just learn Rust! - I don't think you can get around this point by just hand-waving "don't worry about it; someone else will fix the Rust code". Maybe in the very short term, but not in the medium term.
Linus will release a broken Rust kernel. Literally that has happened before in an indirect way because it was the holidays and a patch was not merged, but was explicitly told that the only reason why it wasn't merged was because the holidays made tracking the issue harder.
Also, every new piece of code has to enter linux-next.
There was indeed some confusion about this, because the normal rule of kernel development is that if you change an API, you also fix the downstream users. (A standard-ish monorepo rule.)
However, because R4L is an experiment, the rule seems to be different: it's up to the R4L team to fix the Rust side if things break, so the upstream C devs don't need to care about Rust if they don't want to. (And conversely, they can't then weaponize their reluctance against Rust. That blade cuts both ways!)
However, apparently there wasn't sufficient clarity about this rule. Well, now there is.
In the end, he explicited that not wanting to touch Rust goes both ways: you don't want to touch Rust? Fine. But you lose all rights to complain about what others are doing with Rust.
In other words: don't be a jerk, and don't have selective involvement.
e.g. what they really are doing, as Christoph explicitly admitted in the thread, is that they are attempting to relitigate whether you can use Rust at all in Linux, and convince people that it's so hard to work on that they just give up and go home.
- Linus
In that sense, there's real value in letting a situation develop its own consensus as the actors involved communicate and complete discovery of the factors and motivators involved in any situation, especially if it's clear that the actors have yet to explore that space.
> I was hopeful, and I've tried to just see if this long thread results in anything constructive, but this seems to be going backwards (or at least not forwards).
So probably just hoping some people on either side of the issue would see the light. But he also probably shouldn't have chimed in earlier then.
However they have opinions about how people are interfacing with their code for DMA from Rust, and the rust developers are maybe bent out of shape about it.
Linus is saying: If you're not doing it, then you don't get to have opinions.
But eventually he would have to, if those rust interfaces becomes part of kernel. The entire drama is about DMA guys not wanting to deal with this in future.
I run eBPF code written in Rust in production, and oh boy how more readable it is compared to C. Even simple stuff like `if let Some`, actual sum types, generics, are enough to warrant using Rust over C. And its endtrypoint is basically unsafe { ... }.
We write XDP apps for custom dataplanes where traditionally DPDK would be used (routers and such). Our upcoming network acls are written this way, so close to your netfilter usage.
I do worry about running so much stuff in kernel space though. Imagine a widespread 0day that hits the kernel, or kernel panics causing kernel crashes that require reboots, the user space priv escalation, etc...
Are you doing fail-open or fail-closed? I've been on the fence on that.
- a cloud provider writing a dataplane using ebpf does not have to provide the source - a company selling on-prem software (e.g. esxi) that ships with ebpf probes would need to distribute their source
(note: IANAL)
https://chadaustin.me/2024/10/intrusive-linked-list-in-rust/
So yes I agree that I don't get safety benefits. However, it does not mean I don't get increased reliability from it. Just type safety allows APIs to be expressed in ways you cannot hold them wrong. Heck, just getting a Result<u32, u32> instead of a i64 for faillible operations is a godsend.
In the end, this is why I'm a big proponent of Rust for many areas of programming, including areas where memory safety is the least of your concerns. My gRPC APIs are written in Rust, my system daemons are written in Rust, my eBPF probes are written in Rust. Rust is a modern language whose design is deliberate on many levels to address common issues in programming. If you'd ask me, Rust's "marketing" putting memory safety first is quite a disservice as there are many areas where it helps writing correct programs.
Greg KH noted this in his email in another branch of the thread, and I wholeheartedly agree with him.
Generally, people talk about type system benefits and whatnot on top of safe languages. If you can't ensure safety, UB occurs and that goes out the window. I can't tell how you use unsafe in absolute terms, so I will just hope that you are saying you have an unusually high proportion of unsafe code but still manageable. Otherwise it invites the same "look over everything to make sure nothing can go wrong" practice as with C.
> In the end, this is why I'm a big proponent of Rust for many areas of programming, including areas where memory safety is the least of your concerns.
Memory safety should never be "the least of your concerns". The movement around memory-safe languages is because memory safety is the bedrock on which orthogonal concerns must be placed. Unsafe Rust is well and good because it is (hopefully) used sparingly and with great caution. It is the typical absence of unsafe Rust that characterizes its strengths. I agree that Rust has decent, if not strong, qualities for many other programming sectors, and many Rust critics do make an unreasonable amount of noise about memory safety and unsafe Rust, but Rust is what it is first and foremost because of its approach to memory safety, and everyone can agree to that. Your first comment undermines your stated desire to communicate that Rust should be seen as offering "memory safety plus more goodies".
>> Even unsafe rust is safer than C. Error management, type safety, modern language, etc. There are many reasons to use Rust in place of C, even if your whole code is one giant unsafe blob.
only make sense in the context of eBPF or similar verification environments. That was made more clear in their second comment, so I'm really sorry about that.
>> In the end, this is why I'm a big proponent of Rust for many areas of programming, including areas where memory safety is the least of your concerns. My gRPC APIs are written in Rust, my system daemons are written in Rust, my eBPF probes are written in Rust.
To my understanding, it is only eBPF out of these that has this particular "write all the unsafe you want" quality. Overall, we were on the same page about Rust's guarantees and features this whole time, but I do think @tuetuopay's messaging/"marketing" was pretty confusing, and doesn't help with Rust's marketing that they pointed out.
Even Zig is safer than unsafe Rust.