The only technical problem I can think of is that Rust may not be available for all CPU architectures Linux supports, but this is just speculation on my part having done no research on the matter.
See: https://lwn.net/Articles/862018/
The issue of Rust's LLVM based compiler toolchain not targeting all CPU archs is intended to be solved by the gcc-rs project.
See: https://lwn.net/Articles/871283/
I think the approach the Rust for Linux project is taking is wise: Not about outright re-writes but more about focusing on those subsystems where Rust's intrinsic safety and security properties helps the most.
Your ordinary userspace apps use std (the standard library) which relies on the alloc crate, and that provides heap allocation which indeed panics if the allocation fails. Because it correctly guesses that your "clever" strategy to handle allocation failure actually isn't and will just triple fault anyway so it should cut to the chase.
The kernel obviously doesn't have std, and Rust for Linux implements its own alloc crate.
Linus' requirement that you can fail memory allocation just means all the calls in alloc that can actually allocate memory now return Result to indicate whether the allocation was successful. This isn't how you'd do things in userspace, but, this isn't userspace so fine.
Naive me - not a kernel dev at all - would argue that returning Result<Memory, AllocationError> is always better, even for userspace because it would allow me to additionally log something or gracefully deal with this.
Even if I don't want to deal with it, I could just `.unwrap()` or `.expect('my error message')` it.
Note: I am not trying to be snarky here, I genuinely don't know and would like to.
If answering this is too complex, maybe you can point me in the right direction so I can ask the right questions to find answers myself? Thanks in any case!
If you don't have any memory your allocations are all failing. When you assemble the log message, the allocation needed to do that fails. Bang, double fault.
Now, often people don't really mean they want allocations to be able to fail generally, they're just thinking about that code they wrote that reads an entire file into RAM. If it was a 100GB file that would be a bad idea. But the best answer is: Guard the allocation you're actually worried about, don't ladle this into the fast path everybody has to deal with on every allocation.
Like sit down, figure out all the things you'll want to do on an allocation failure, and once you have determined that you slice a little chunk of memory when you start your app (and maybe _that_ fails and you can't do anything). and when you hit a failure you do your think, then tear stuff down.
Now that I see interactive programs taking a gigabyte and the system being ok, I guess we're in a different regime.
In general Don't assume anything about your global process state just because one allocator fails.
It never occurred to me (being in non-embedded land) that returning an enum as the error or a &'static str instead of a heap structure like String, could also fail.
Seeing that Result isn't part of core, but of std, this makes sense.
Just to tickle my nerve though: theoretically speaking, with your example, it would work, right?
I couldn't allocate 100GB (because OOM or not even enough RAM to begin with) but it could be that the system can allocate the needed memory for error message just fine.
Very interesting.
Add to that that several operating systems will lie about whether you're out of memory, so the 'error' or failure will often not be on the Result() value but come in a SIGKILL instead, it's just adding complexity.
People that are actually worried about it and no how to deal with it, will be coding with a different style and can use the alloc library where/when they need to. (at least when it gets stabilized in Rust)
I've never checked core before, so I did when checking up for this discussion.
I somehow missed Result. Silly me didn't search on that page, but ofc I found it on std
https://doc.rust-lang.org/std/result/index.html
Also thanks for clarifying that values of Result can be stack-only!
> I could just .unwrap() or .expect('my error message') it.
Panicking can allocate. Allocating can fail. Failing can panic. Panicking can allocate. Allocating can fail. You can bite yourself in the ass like a real Ourobouros.
IMO, a prerequisite to using fallible allocation APIs should be attempting to write your own allocator, handling the weird and wacky problem of initialising a data structure (for the heap) in such a way that if it fails, it fails without allocating but leaves some hint as to what went wrong.
So for `Rust for Linux` they also need to implement a `Result-like` type that is stack only based to solve this issue, right?
If so, cool, thanks, you just made my day by tickling my learning nerves! :)
It actually has to do with `panic!(...)`. When you use `unwrap()`/`expect("...")`, you use the panic macro under the hood; parts of the panicking infrastructure use a boxed trait object which could contain a static string or formatted String or anything else really. The box can allocate if it is not a ZST. I believe the alloc crate's default handler tries to avoid this kind of thing, so that it can't fail to allocate AGAIN in the failure-handling routine. It will likely do a better job than you could.
This is a live issue at the moment, so to go into any more detail I'd have to read a bunch of recent Rust issues/PRs.
They can kill entire misbehaving processes. What are you going to do in your little program, clear a cache whose objects are sprinkled evenly across 150 different pages? You would need more control than you get from blindly using malloc/free/rust_alloc globally. Something like memcached would be able to use these APIs, because it uses its own allocator, and knows enough about its layout to predictably free entire pages at once.
That isn't very accurate. In Rust when programming in no_std, you can (must?) define your own panic handler:
https://doc.rust-lang.org/nomicon/panic-handler.html
Which you would define in the kernel. While I'm not going to speculate on exactly what the implementation would look like, you definitely do not need to "crash" the computer. I haven't done any kernel programming, but I'm guessing the kernel could do some things at that point with shared memory space that is already allocated to deal with this situation and try to recover in some way.
Edit: for example, I just found this in the kerla project: https://github.com/nuta/kerla/blob/88fd40823852a63bd639e602b...
That halts now, but it probably doesn't need to, or could do it conditionally based on the contents of PanicInfo.
> panic handler [...] Which you would define in the kernel. While I'm not going to speculate on exactly what the implementation would look like, you definitely do not need to "crash" the computer.
The panic handler loses so much of the context that crashing the computer is the only thing you can practically achieve. You can't retry an operation generically from with a panic handler, it doesn't know anything about the operation you were attempting. The OOM handler gets a Layout struct only. You could try unwinding or something, but within a syscall handler, I don't see how anything good can come from that. Unwinding in the kernel is simply a terrible idea. What else are you going to do?
Now there is a lot that the allocator could do. If you wanted something to be retriable, it could be interesting if the thing that failed was an async task. If so, that panic info could carry enough information to say, the failure was an OOM, here’s the task that failed, and it is marked as retriable. Yes, this would require a store of tasks somewhere in the kernel. Then based on it being an OOM, see if any memory can be reallocated before retrying, or wait to retry until it is.
This is where theoretically a new async based Rust kernel, especially a micro-kernel, could be interesting. Is stack unwinding in the kernel a bad idea? Maybe. Can it be done in Linux? Maybe not, maybe it’s too much work to track all this information, but I disagree with the conviction with which you right it off.
The Error type currently isn’t in core, but for other reasons, that just got resolved: https://twitter.com/bascule/status/1452029363197784071?s=21
We have a light / temporary fork of the Rust stdlib allocator with fallible allocation support: https://github.com/Rust-for-Linux/linux/tree/rust/rust/alloc
See e.g. https://github.com/Rust-for-Linux/linux/commit/487d7578bd036...
If the kernel is running out of memory, IMHO that's a bug. The kernel is ultimately responsible for memory management right? It needs to prioritize itself over everything else or the system is in trouble.
The limitation of that approach is that all internal interfaces (which includes most data structures) have to be C-compatible. You can get much of the safety benefits of Rust, but lose a lot of Rust's ergonomics.
Just like Android Linux compiles just fine with clang for the last five years or so, it now makes use of Rust.
https://source.android.com/setup/build/rust/building-rust-mo...
If upstream ever cares to support clang or Rust, that is another matter.
C rewritten in Rust is still very C-like, and requires refactorings that you can't do until it's all Rust.
BTW: There are two GCC-based Rust implementations in the works, so compatibility with exotic platforms is going to be solved.
I still think the project is really cool and worthwhile.
And then, in the longer term, someone ought to write a book about operating system implementation in Rust that ignores Linux interoperability and focuses on readability, maintainability by showing use of Rust's idiomatic style.
It’s just not an interesting or unique perspective, and pretty silly to suggest seriously.
Using an incremental approach at least gives us some benefits in the near future.
I'm not even saying a fresh start would be a bad idea. But incrementally replacing parts of Linux seems like a more promising approach, IMHO.