Writing an OS in Rust: Remap the Kernel
os.phil-opp.com
os.phil-opp.com
I started it because I was unhappy with the existing documentation. Most resources were mainly for 32-bit and there was hardly any documentation about Rust OS development. One day I decided to do a clean rewrite of my experimental rust kernel and had to look up all the required information again (since I had forgotten the details). So I decided to create the missing documentation myself and write some tutorials about booting a Rust kernel.
I wrote the first posts over the summer and it was kind of fun. I never wrote a blog post before and English is not my native language, so it was pretty challenging. I spent a lot of time rewriting and improving them until I was more or less satisfied. Eventually I had the guts to post it in the rust subreddit. I never expected that it would be useful for many people (who would want to write an OS in Rust?), but it totally took off. Someone posted it on hacker news and it even was the top post for a short time. I was completely stunned :D.
I really love writing these posts. It takes much time but it's a great way to really understand the code since you need to explain every detail. It also helps to make the code as simple and straightforward as possible.
I still don't know where this will lead. But I will keep on writing as long as people want to read it :). For the short term the plan is to explore interrupts and to add keyboard support. After that, I'm open to new ideas :).
Doing it in a fresh, up-and-coming language like Rust is icing on the cake. Keep it up!
It looks like there are a lot of fast moving projects on this now[0], namely Redox[1], and a lot of research and study demo codes. Looking forward to seeing how these efforts come together.
Just wondering why Rust/D get lumped into OS and low level development in general when it seems more syntactic than anything else. Do you have to manually manage memory in Rust?
If you don't understand why that is critically needed, I'd like to introduce you to /almost every RCE, buffer overflow, and stack smashing bug/ that has plagued us for the past 20 years.
Very secure and reliable operating systems have been written in existing languages such as ESPOL (MCP) and PL1 (MULTICS). There's also Zetalisp (Genera) and Oberon (Oberon), though they might not appeal if you dislike garbage collection. You might also consider Ada, designed with type and memory safety in mind and used in defence and avionics.
The problem with these older systems is that they don't support heap allocation while remaining memory-safe. (A quick Google search seems to confirm that MISRA-C is in this category.) Or, if they do support heap allocation, they use a GC to manage it. Both of these restrictions are often considered undesirable for a kernel, except in specific embedded situations in which you can get away with statically allocating all your variables. Rust's lifetime system can ensure that you use malloc/free correctly, so it doesn't have to wall dynamic memory allocation off into unsafe code.
Yes.
Building an OS rather than applications still has some rough edges, but we're working on them. One of the reasons that I'm playing with OS stuff is to make sure we have a core team member to advocate for those features, as they are fairly niche. Most people are interacting with their OS, rather than writing one, so it's easy to put them off.
The fact that is has better syntax is almost anecdotal. Same story for Rust.
¹: Well, you can mess up in `unsafe` blocks. It's basically an “I'm smarter than the compiler” annotation. You can use it when Rust can't guarantee that something is memory safe. For example writing to the VGA text buffer at the special address `0xb8000` is totally safe, but Rust can't know that. We try to minimize the number of `unsafe` and to double check each use, but of course you can make mistakes and accidentally violate memory safety. But even then you just need to look at `unsafe` blocks to find the bug, which is still a big improvement over languages such as C.
This is often said about Rust, but isn't exactly true. If you'd like this to be true in your own code, you have to have be discipline and put all operations which may violate invariants required by unsafe code inside unsafe blocks, even if the compiler considers that code to be safe.
Imagine you're implementing a type like Vec<T> that has a (raw) pointer 'buf: *mut T' to some slab of memory it owns, and another field 'length: usize'. A safe method of Vec<T> could make a mistake while mutating the length field and violate memory safety in a completely correct 'unsafe' block that assumes '[buf, buf + length)' is a valid range of memory.
Tracking a semantic error in a call that's three calls away from where your unsafe block lives is much easier and structured than going through everything everywhere for any error from anything.
(I haven't done much work at all in Rust and avoid `unsafe` - and `mut` and `.unwrap`, for that matter - like the plague when I do use it, so I'm very open to being wrong in my interpretation.)
EDIT: I also think a reasonable comparison would be Haskell's `do` blocks. It's guaranteed that anything with side effects will live in a `do` block, but of course you can still have semantic errors. Again, you know exactly the "functions of interest" to look at if this is the case - there's no chance of `someFunc1` to be doing anything funny if it's never called by anything in the `do` block.
For some definition of "have to" and "manual".
Rust uses RAII and ownership/borrowing for safe, performant memory management (no GC). At the same time, it doesn't automatically use memory. So, stuff goes on the stack unless I explicitly put it inside a heap-based thing (e.g. a box or a vector). These things clean up after themselves and can be used cleanly.
Like Sam said here (https://news.ycombinator.com/item?id=10711997), "You have to think about it. You don't have to worry about it."
At the same time, since you don't have to use the heap, and low level code isn't too hard to write in Rust, you can easily write non-allocating OS-level code if you wish. And with the appropriate (unsafely implemented, but verifiable) zero-cost abstractions, you can still have safe code at this level.
AFAIK it will roughly follow the first posts of Writing an OS in Rust, but at a much slower pace and with more extensive explanations. After the Set Up Rust post, it will diverge from the series and explore keyboard input instead of memory management.
Steve is a dang awesome writer (he wrote most of the Rust book) and intermezzOS looks pretty great already. I think it will bring new people with new ideas into OSdev and thus I'm really excited about it.