Writing an OS in Rust: Better Exception Messages | Hacker News Reader

Writing an OS in Rust: Better Exception Messages

os.phil-opp.com

220 pointsdikaiosune31 comments

Pretty cool that this was done in pure Rust, although it might have been easier to write a small trampoline in assembler code that calls into Rust code.

This is what I did in a hobby kernel project of mine [0]. This was pretty simple to do and easy to do all interrupt handlers at one go, leaving the actual logic to C code. The downside is that it uses unnecessary amount of memory (at least 16 bytes per interrupt vector).

The Linux kernel has a similar but more clever solution where the actual interrupt vectors are tightly packed and the number of interrupt is determined with arithmetic from the instruction pointer. I can't find a link right now, if anyone finds it, please post a link.

Side note: the use of Intel syntax is an annoyance. You can write inline asm in Intel syntax but it's not exactly the same as nasm/masm style and you still have to read at&t syntax in objdump output (by default at least). I prefer sticking to at&t when working with GNU tools. Even if you prefer Intel, it's still better to have just one syntax.

[0] https://github.com/rikusalminen/danjeros/blob/master/src/arc...

"although it might have been easier to write a small trampoline in assembler code that calls into Rust code."

That's what he did; it's just embedded inside Rust code.

This is the easy part - capturing exceptions. Handling them, and returning control with the machine state intact, is the messy part.

(It's late to be writing an operating system with demand paging. The most paging ever accomplishes is the illusion of 2X the RAM. RAM is cheap today. The price you pay for paging is random large delays in your program. Is it worth it to save some RAM cost? Mobile devices and real-time OSs don't demand-page.)

> This is the easy part - capturing exceptions. Handling them, and returning control with the machine state intact, is the messy part.

Yeah, I plan to tackle this in the next post: Compiling without red zone, saving/restoring all relevant registers, and disabling SSE instructions in the kernel to speed things up.

> (It's late to be writing an operating system with demand paging. The most paging ever accomplishes is the illusion of 2X the RAM. RAM is cheap today. The price you pay for paging is random large delays in your program. Is it worth it to save some RAM cost? Mobile devices and real-time OSs don't demand-page.)

I'm not sure if I really want to implement demand paging, I've just used it as an example. In the next post we will only implement lazy page mapping, so that we no longer need to preinitialize the complete kernel heap.

I would guess that paging could be used for other purposes though, such as compression of inactive application memory.

I worked with an embedded phone OS once where the entire realtime OS ran as a single task in an L4 microkernel.

This was to allow the realtime OS to run in a demand paged environment: if the OS accessed an address from the area defined as 'rom', L4 would allocate a page of RAM and populate it from flash.

The whole purpose of this was solely to allow them to use a cheap NAND flash chip rather than a NOR flash chip (NOR flash can be memory mapped, NAND can't). That was the only thing they used L4 for. Everything else operating-system-wise was done by the embedded OS, device drivers and all.

Surprisingly, it actually worked. (The rest of the OS was a horrible trainwreck, but not this bit.)

> I'm not sure if I really want to implement demand paging

Yes, you probably should. It's the quintessential example of using virtual memory and handling page faults so you probably want to do it just for the educational value (although that would need a disk driver too).

But it's also the key to implementing various modern operating system tricks, such as memory mapped files and dynamically expanding (and contracting) stack (for userspace processes, kernels typically don't do this AFAIK).

What you probably do not want to do is old-fashioned swap space on disk. It's not as useful as it once was because we have A LOT of memory these days and it might be harmful for SSD wear. For implementing this, you'd also need to have some kind of LRU cache to figure out which pages can be swapped out (quite typically in applications like this, swapping in is the easy part, figuring out what to swap out is more difficult).

> What you probably do not want to do is old-fashioned swap space on disk

Yeah, that's what I meant. I definitely plan to use the various lazy loading tricks for mmaped files, loading executables, etc. But first we need a disk driver :D.

One advantage of not supporting paging is that it gets the disk driver and file system out of the kernel. They're not needed early. You need a boot loader that can load the OS and process images, so the OS starts with some processes running. The disk driver (if any) comes in that way, rather than as part of the kernel.

QNX works like this. It's convenient for embedded systems, where you may have no disk at all, ROM, flash, or a hard disk.

> That's what he did; it's just embedded inside Rust code

The interrupt handler is a naked Rust function (really cool language feature btw), not an assembly trampoline. There may be a line or two of inline asm in there.

This is probably more educational and experimental than doing it with asm like every other OS project does. I thoroughly enjoyed reading about it.

On-demand paging is still relevant, it's used for mmap and memory mapped GPU buffers, etc. Plain old swap areas may be a thing of the past, though.

Cool stuff, what's the significance of "!" as a return value? Haven't seen that one before.

apparently it indicates divergence, i.e. the function does not return at all. Very interesting, I've never seen the notation before.

See https://doc.rust-lang.org/book/functions.html#diverging-func... and https://doc.rust-lang.org/reference.html#diverging-functions

Swift encodes this with a noreturn attribute, and I think it's a "standard not standard" C attribute as well.

(in case you were wondering who else does this)

_Noreturn is a standard keyword in C11. C++11 has a standard attribute called [[noreturn]].

It means that the function never returns (via exiting the process, exiting the thread, or by panicking). It's useful because the result of a function that returns ! is compatible with any type, so you can write things like:

    fn usage() -> ! {
        println!("usage: myapp FILENAME");
        process::exit(0)
    }
    
    let filename = if args.len() < 2 {
        args[1].clone()
    } else {
        usage()
    };

Good to know, that explains why I've never seen it.

And an RFC for it to become its own type recently got approved: https://github.com/rust-lang/rfcs/pull/1216 (The github thread also has a ton of good discussion about it. I learned a lot).

Up until now it's been a big edge case. This makes it considerably smaller.

Is ! verified or enforced; i.e. does the compiler check to make sure that the functions does terminate in some way (besides returning) ?

Looks like it [0]. The author had to use inline assembly, so compiler couldn't enforce it and failed to compile. They had to add an annotation to say that the code after the assembly was unreachable.

[0] http://os.phil-opp.com/better-exception-messages.html#intrin...

It's the exact same analysis as ensuring all paths of a function do return the right type -- you just make sure all paths call a divergent function (typically unconditional panic) or enter a loop {} that never breaks/returns.

This is of course a conservative analysis so e.g. "while true { }" is assumed to terminate and is rejected.

Failing to perform this verification correctly would trivially lead to memory unsafety as calling code is allowed to assume such a function returns any and all requested types.

It'd be type and memory unsafe if it weren't verified, so yes, it is.

It seems to be equivalent to Haskell's Void type. The Void type is simply the type of no values. If you cannot create a value of this type, yet you "return" it, then it is proof that you never return.

Also "Nothing" in Scala and "never" in TypeScript. The bottom type[1] goes by a lot of names.

Interesting, I think Haskell does sort of have a value for it, undefined. It's just that if you look at that value too hard, it throws. :)

[1]: https://en.wikipedia.org/wiki/Bottom_type

See: https://en.wikipedia.org/wiki/Bottom_type

Does Linux have exceptions?

All CPUs with MMUs use them, so Linux has to handle them.

umm, that's not true. Not all CPUs with MMUs have hardware support for memory based exception handling often the job of a MPU (memory protection unit). The ARM Cortex-M4 says it's optional [1] and as such Ti Jacinto6 processors don't implement it.

[1] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc....

The Cortex-M4 does not have a MMU; there are no page tables or anything. The MPU you mention is a sort of primitive MMU.

It does on the Ti Jacinto6. In fact there are two MMUs. Cortex-M4 MMUs are implementation specific. The Jacinto6 (aka J6) M4s are housed in what they call an IPU. Each IPU (2 of them) have 2 Cortex-M4 cores for a total of 4. Each IPU has an IPU MMU (called an AMMU) and they also can access memory through the L3 MMU; the same that the A15 cores of that chip use.

So when you say the Cortex-M4 does not have an MMU, you're categorically wrong because it can and does have one on Ti J6 chips.

The whole blog, at http://os.phil-opp.com, is phenomenal.

Thanks so much!

Reply on news.ycombinator.com