Writing an OS in Rust: Catching CPU Exceptions
os.phil-opp.com
os.phil-opp.com
>The implementations of these methods need to modify the correct bits of the u16 without touching the other bits. For example, we would need the following bit-fiddling to set the stack index:
self.0 = (self.0 & 0xfff8) | stack_index;
self.0 = (self.0 & (!0b111)) | stack_index;
self.0 = ((self.0 >> 3) << 3) | stack_index;
self.0.set_range(0..3, stack_index);
Personally, I disagree. Whilst it is easy to make mistakes when writing bit-fiddling code (there seems to be one in the second example, unless Rust has on of the more weird !-operators in programming languages), or any code for that matter, the meaning of all the bitwise operations is easy to recognise.
Without knowing the API, the transformed line rings no bells. How many APIs do you want to learn?
[1]: https://doc.rust-lang.org/book/iterators.html [2]: https://doc.rust-lang.org/nightly/std/ops/struct.Range.html
For example, what happens when I specify a range of 0..100 for a 32 bit int? what if the value is too big for the range? APIs around small operations like this introduce ambiguity that wasn't present before.
So what? You could implement addition using bitwise operators, but it makes for much more readable and less error-prone code to abstract it into an addition operator. Code should clearly express intent, and if your intent is to set, say, bits 5 though 9 of some register to some value, the clearest expression of that is something like (given a right-open range convention, which is well-established in this particular language):
register.set_bits(5..10, value);
register = (register & 0xFC1F) | (value << 5);
>For example, what happens when I specify a range of 0..100 for a 32 bit int?
What happens is that you've violated a precondition of the API. Because this is Rust, I would expect the operation to be implemented in such a way as to panic (in debug mode), or just fail to compile, since selecting a bit range like this at run time is extremely uncommon.
>what if the value is too big for the range?
Same thing as if you weren't using the abstraction around the bitwise operators: either you'd clobber higher bits, or the implementation would truncate the value.
Again, though, I would expect a debug-mode panic.
>APIs around small operations like this introduce ambiguity that wasn't present before.
I'm not sure what "ambiguity" you see here. There's the issue of whether the range is right-open or -closed, but again right-open ranges are well-established in Rust.
As for the bit ranges, it looks backwards to me, since bits are normally numbered high to low. I would find it easier to read if it read:
self.0.set_range(3..0, stack_index);
Luckily Rust gives you the #[test] directive so the next line after abstracting away the bit-flipping details you can provide it works correctly.
self.0.stack_index = stack_index;
This best represents what you're actually doing, instead of faffing around with bit twiddling to achieve the same result.
Does Rust not allow something like this?
There should be. Obviously, unions with pointers are special. Some code might need to be marked unsafe. I suspect there is a subset of pointer-in-union operations that could be made safe, possibly either changing or using the pointer but not both.
Another thing missing is bifields. I'd love to see this done right, with adjustable packing so that one can use them to write an emulator or file parser.
I'm not going to develop while connected to the internet. This is not allowed by any employer that cares about security.
I prefer to use the package manager that comes with my OS. Every other installer (firefox, nvidia crap, etc.) risks screwing things up.
Given the two things above, I hope you can see how much of a pain it would be to run some non-standard installer. I'd need to ask IT to mirror the rust cargo stuff... and is this even possible? Would I have to dig deep into the rust stuff to change a public key?
With it being a crate, I worry about it going unsupported. I've seen this all too often with firefox extensions. If not that, maybe somebody will come along and change the API.
I couldn't even find the crate. I found one called bitflags that does 1-bit fields. That isn't suitable. I'm looking for something that can handle stuff like an x86-64 descriptor table (GDT, LDT, IDT) entry or a PowerPC opcode. The PowerPC mtspr and bl instructions are interesting examples: mtspr has a 10-bit split bitfield, with a pair of 5-bit halves in the wrong order, and bl has a field that kind of has the low 2 bits stolen by other fields. The x86-64 descriptors have lots of strange-sized split fields and they are tagged unions for which the tag layout is hardware-defined. Another good one is page table entries, from the perspective of: emulator, hypervisor, OS.
With this not being part of the language proper, I have to wonder how well it performs. Has anybody compared the resulting assembly code?
> I'm not going to develop while connected to the internet.
> ... and is this even possible?
> I couldn't even find the crate. ... that isn't suitable.
> I have to wonder how well it performs.
Also, finally, you don't have to use crates.io to use Cargo. You can just specify crates that are on your local filesystem, or a git repository hosted anywhere.
I suppose it could be that it's simpler to write it out in assembly, since the C (/etc) code would look basically the same - just a lot of assigning cryptic variables to cryptic pointers?
Or, based on a quick Google search (http://stackoverflow.com/questions/3022046/is-it-possible-to...), is it because you cannot write to these registers from C, thus needing to drop down to assembly?
SYSCALL is even worse -- when the handler is invoked, there isn't even a stack.
Unfortunately, on x86, privilege transitions and exception handling is an incoherent mess, and, worse, privilege changes (via IRET) can fail. Most kernels messed up what happens when IRET fails for obscure reasons, resulting in corrupting the value of GSBASE and therefore allowing a malicious program to change the targets of certain kernel memory accesses.
The result was "BadIRET", which I discovered, fixed on Linux, and reported via CERT (not a good experience) to other vendors. Someone else named it, and a few other people wrote exploits based on it.
1. As you'd pointed out, the language in question might not let you generate certain instructions. For instance how do you express LGDT in C? You can't. You'd instead write it in assembly (or inline assembly) and expose it to the C world as a function (or in a compiler that supports it, an intrinsic) that it can call.
2. A CPU's understanding of state is defined around the notion of registers and memory. A programming language however exposes a higher level abstraction. For instance, a function call in C is an abstraction that defines both transfer of control (which the CPU knows about) and persistence of local state (saving and restoring registers, stack pointer etc., which the CPU knows nothing about). This is called the ABI and is merely a previously agreed upon convention. Higher level languages (say Java) are able to provide richer abstractions than merely an ABI as their "agreed upon conventions" are defined at a much higher level than in terms of a CPU (registers and memory). Java, for instance even defines a virtual architecture in terms of which facilities offered in the language are defined. User code written in one of these higher level languages depends on these facilities being available to them at any time. This is what makes it unsuitable for writing certain sections of OS code. For instance, the CPU expects the OS to save and restore register state around an interrupt handler. If said handler was written in C (as a function) though, it'd expect its first six arguments in certain registers etc. - conventions, the CPU knows nothing about. You can think of the CPU as exposing its own, simple ABI which can only be implemented by writing certain sections of code in assembly (This is not entirely true though, as one might be able to mimic something like this in certain compilers, especially C compilers, that let the programmer write "naked" function bodies and custom prolog / epilog code). This is also why most parts of the OS, that don't directly interact with the CPU, can and are written in a higher level language. For e.g. The Singularity OS by MSR was written in C# and assembly.
You mentioned: "For instance, a function call in C is an abstraction that defines both transfer of control (which the CPU knows about) and persistence of local state (saving and restoring registers, stack pointer etc., which the CPU knows nothing about). "
Doesn't the CPU know about this "persistence of local state"? I mean it pushing and popping that state form the stack right? I'm curious what you mean there.
What's a "naked function" in this context? I think of naked function as one that doesn't have a return statement.
Cheers.
The CPU doesn't know about the persistence of local state in that it doesn't know (in this case), the significance of anything that is being pushed onto / restored from a frame around a call. It knows that a "PUSH R9" writes the value of R9 onto the region in the stack being currently pointed to by the stack pointer (RSP). It however doesn't know that this is being done because, the current frame has a live value in R9, which, per the ABI, the function being called is allowed to trash, as it is considered a volatile, callee saved register. Like I'd said earlier, these are just agreed upon conventions that might even change across compilers.
I'd used the term "naked" the way the VC++ compiler uses it. A "naked" function is one without a prolog or an epilog [2].
[1] https://pdos.csail.mit.edu/6.828/2008/readings/i386/LGDT.htm
The problem is that we're in 32-bit mode at the beginning. So we would need two Rust libraries, one for the bootstrap 32-bit code and one for the 64-bit kernel code. We would need to link them together somehow or use GRUB modules. Doing it in assembly was somehow simpler and more fun :)
So let's assume you wanted to create an OS with a minimal amount of assembly: The minimum requirement of all higher level languages is at least a valid stack pointer. On x86, you need to initialize the stack pointer explicitely through a `mov esp, 0x{stack_pointer}`. It might be possible to do it in a naked Rust function [1]. As soon as you have set up a stack, you can directly call into Rust or C code (see [2]).
It's easier on ARM Cortex-M boards, since the stack pointer is loaded from address 0 on reset. So you can boot directly into Rust, without any assembly file (see [3], especially the `common.ld` linker script).
[1]: https://github.com/rust-lang/rfcs/pull/1201
Signals have the same issue in ordinary Rust programs. See [1]. What's the state of the Rust world when you're in a signal handler?
I don't think signal handlers are a big deal because global mutable state in rust is rare; and when it exists it probably is immune to the issues -- global mutable state in rust is required to be thread safe (since rust doesn't know if you are using threads), which should make it signal -safe too.
It interesting that they stripped down the runtime from previous(larger) iterations. Are those discussion discussed anywhere that you know about? It seems like that would be an interesting read.
There are also old mailing list discussions on rust-dev somewhere which discuss this IIRC.
In the past, Rust did have a bulkier runtime with a green thread scheduler and libuv; a runtime that "mattered".