Writing an OS in Rust: Advanced Paging

That said, there is a lot of room to innovate in OS/userspace boundaries that don't rely on POSIX APIs--see epoll/kqueue/iocp for one place where this innovation already occurred. I'd love to see better alternatives to POSIX signals, for example. So an OS that treats POSIX as a mere compatibility layer and not the primary driver of its design could be welcome.

You should encourage people to make native software using the new paradigms and abstractions of the OS, built from the ground up with decades of hindsight, rather than encouraging crappy ports.

> You can only get adoption of a non-POSIX OS if it offers something really unique to the user

At this point, just not being built on 50 year old crappy standards would be pretty unique. I have my own personal wish-list of features that would lure me in even if the only hardware supported was an RPi.

Then who cares about the OS? Why even bother. Just use ChromeOS.

https://github.com/redox-os/tfs

I wonder how that effects the time horizon for maturity. I've always just assumed that a filesystem takes a good 10 years to be really stable. Maybe my assumptions are way off? Or the facts that its a reimplementation makes that matter much less? Didn't it take BTRFS around this length of be considered production stable?

And I'm pretty sure Windows has some C++ in kernel space. It might be limited (no exceptions etc), but that's another matter.

Or concurrency safe. There aren't many languages that can help with this class of bugs. Like race conditions. (Someone on HN said Pony is another language that has concurrency/thread safety guarantees.)

And I'd wager that C++ can write far better optimized abstractions than a human in C every could. In most cases.

Writing memory-safe C is clearly possible - many other enviroments are written in C. It come down to using a safe subset.

The main actual objection to C++ is that, well, the exception handling model is controversial, and it is especially ill-suited to kernel programming. RTTI can also get thrown in the lump here. Standard libraries aren't going to work in the kernel anyways, and so if you strip that out, what benefits does C++ get you over C? The correct answer, actually, is that you get RAII for added safety, and tooling such as IDEs have better support for finding all implementations of a virtual function than they do function pointers, but for a lot of C programmers, this can be a sense of "real programmers shouldn't need such assistive technology."

What? No... the most useful parts of C++ are its compile-time facilities... templates and type utilities and RAII/smart pointers/etc. (I could go on and on) all of which are still there. You'd lose some runtime stuff like the ability to throw exceptions and some RTTI, which some people already advise each other to avoid in user mode too (not to say I agree, but they're still finding a lot of use in C++ without this). If losing those is equivalent to losing most of C++ for you then you're not really taking advantage of C++ to begin with.

This is commonly accepted wisdom, but it's not entirely true. (It's only a little bit true.)

The kernel is a freestanding environment where big chunks of the standard library are missing. However, the most useful parts, like unique_ptr<T>, can be reimplemented as needed.

The kernel absolutely can support RTTI and exceptions. Porting libcxxrt is relatively simple, for example, even if implementing the entire Itanium C++ ABI is a big task. Though, the kernel should still avoid throwing and catching exceptions in sensitive code like interrupt handlers. This is not radically different from other C++ projects. The use of exceptions is typically discouraged, for example, in tight inner loops.

It is true that enabling RTTI and exceptions causes code size to explode. This is a valid concern. While memory is cheap, cache misses are not cheap. Mark functions and methods with "noexcept" as needed where this becomes a problem.

Finally, it's absolutely true that exceptions make it impossible for the kernel to make solid real-time guarantees. Fortunately, most operating systems only have soft real-time constraints, if that.

This isn't really true; you still have memory safety, and generics and a sane stdlib.

Those things the compiler is angry about? Those are things that would be runtime errors in C.

`rustc` is pedantic because it needs to be in order to provide the assurances it gives. There is a subset of "valid" Rust code that `rustc` rejects because it isn't smart enough to understand it would be safe, but I would guess that those cases are less common than than the cases of "seemingly valid" Rust code that is in reality problematic if it were accepted.

Beyond that, I would like to hear what errors have made you feel like the compiler "is angry" at you instead of feeling the compiler "is trying to help me". We spend an inordinate amount of effort on 1) accepting code that we should accept by improving ergonomics, 2) provide extensive suggestions when "seemingly correct" code hits the wall of `rustc`'s understanding, language design or things addressable by ergonomics, and 3) provide as good explanations as possible for all other cases. Because of this, I believe the experience of a newcomer today is much better than even a year ago, even though it can still be improved.

First, I would recommend taking a look at how bootloaders like GRUB handles this, since they do it pretty dead simple and make it easy for kernels to know exactly what situation they're going to be in right when they start. For GRUB, when loading a `x86_64` ELF kernel, it loads the kernel's segments at whatever physical address they are linked at (ELF records separate physical and virtual addresses for segments), and then identity maps everything and jumps to the kernel's entry point. Note it doesn't do anything else, like setup a stack, it assumes the kernel will do that if it wants one.

At that point, there's a problem, because almost all kernels are going to be higher-half and thus linked at a different virtual address then their physical address (But GRUB ignored the virtual addresses). Thus, the kernel's entry point needs to have some type of stub which can run at the identity mapped location, create a new page table with both the identity mapping and the higher-half mapping, and then jump to the actual kernel at the virtual location.

Now with that, I want to point out that the kernel does not need to use the page table that GRUB set up, and in fact GRUB doesn't even provide a good way to do that. But they don't have too - the kernel can simply load a new one into `CR3` to completely replace GRUBs page table. And once they do that, they can jump to the higher-half and then remove the identity mappings. And they can do this without having to do any allocation because they can simply make the initial page tables static variables, and then they'll already be in memory ready to be used once GRUB loads the kernel into the correct physical addresses.

You can see that in my kernel here[0], though it's a fair amount different being that it's 32-bit. The same idea applies though - my code is linked at a higher-half address, but GRUB jumps to the entry point while running at the lower physical address. This means that if I were to jump straight to my C code, it would be completely broken, because it was compiled assuming it was running at `0xC0100000`, not `0x00100000`. The assembly stub fixes stuff up just enough so that we can run at `0xC0100000`, and then later in the kernel we setup a more involved mapping.

This gets to another important point that I couldn't really figure out - how is your kernel linked? I looked at `cargo-xbuild` but I couldn't figure out it. I don't quite know what linking looks like for Rust, but I would think at some point you drop down to the LLVM linker and should have some linker script like this[1]. The `AT` specifies the physical address for those segments to be placed at, where the `. = KMEM_LINK` specifies the location the code will run from - that's how you achieve a higher-half kernel. I'm guessing that, if you don't have any linker scripts, then it just gets linked at whatever addresses the linker happens to pick for normal executables, which will cause you problems down the road since your kernel will be located at a weird location.

And now that I'm looking at the code further, I'm guessing you may have already run into this problem, since I noticed in `bootloader`[2] that you ignore the ELF file's physical addresses[3] and instead just load the kernel at a constant physical address. I would highly recommend against this, the ELF kernel should have correct physical addresses and if you ignore them then the kernel will not be able to correctly create a new page table itself, since it will setup incorrect mappings based on the assumption of where it thinks the kernel is located in physical memory vs. where it was actually loaded. The identity mapping will also not working, since the stub to load the new page tables will attempt to use incorrect physical addresses.

This does make the bootloader a fair amount more challenging though, since you have to move the code that loads the kernel into memory into an piece of memory unused by the kernel. But I would say for a first pass, simply moving the bootloader to a somewhat higher physical address and then assume the kernel will be at a low physical address (and panic if it's not) would be fine. Also, you still need to place the `BootInfo` somewhere, but you can just pick any random physical memory location. As long as the kernel knows the physical address of where it is, it can avoid writing over it (Or more likely, the kernel will just copy the entire thing into a separate structure inside the kernel to keep things simple). Either way, the kernel can handle that pretty easily.

> The question is how we can support that approach in the `bootloader` and `x86_64` crates without breaking users who wish to continue using recursive paging.

I don't really have a good answer for you as for avoiding breakage (I think it's somewhat impossible here), but I would argue that any kernels that want to setup their own recursive setup should simply do it themselves. They can write the entry point to their kernel to do whatever they want, including setting up a recursive mapping.

If you have any questions please feel free to ask them! I could also try to condense this somewhat and put it on the `github` issue if you'd like. But to respond directly to the ideas in the `github` issue:

> The recursive_level_4_table feature enables the recursive mapping of a level 4 entry that we currently have. Provides a RECURSIVE_LEVEL_4_TABLE_ADDR constant to the kernel. > The map_physical_memory feature maps the complete physical address space to some virtual address. Provides a PHYSICAL_MEMORY_OFFSET constant to the kernel.

If you respect the physical addresses in the ELF file and setup an identity mapping with them, then neither of these are necessary - the kernel doesn't need the `PHYSICAL_MEMORY_OFFSET` because it already knows it's at the locations specified in the ELF file, and the kernel doesn't need the location of the current page table at all because it can just replace it with a completely new one that also has the correct identity mappings for the current physical memory location.

[0] https://github.com/mkilgore/protura/blob/master/arch/x86/boo...

[1] https://github.com/mkilgore/protura/blob/master/arch/x86/boo...

[2] https://github.com/rust-osdev/bootloader/blob/master/src/mai...

[3] https://docs.rs/xmas-elf/0.6.2/xmas_elf/program/enum.Program...