Writing an OS in Rust to run on RISC-V | Hacker News Reader

Writing an OS in Rust to run on RISC-V

gist.github.com

229 pointsfavourable72 comments

HIGHLY highly recommend checking out Stephen Marz's OS blog, where he walks through building a basic RISC-V OS in Rust step by step: https://osblog.stephenmarz.com/

I’m asking this question as someone with zero experience in writing an OS, but who has done limited services development. Would it be an interesting research project to write an OS with the following traits: 1. The purpose of the OS would be to run microservices only 2. The OS would provide the minimum functionality to provide a platform for services to run on the web assembly system interface 3. Some interim solution for providing sockets would have to be built until wasi spec supports it 4. No file system, no writing to disc, OS gets booted from USB, services pulled and initialized from network

Apologize ahead of time if this is a naive question and I need to jump into more traditional OS dev to get my bearing. I’m a mobile/services taking a break and I just happen to currently be intrigued by Rust and WASM.

You could start your search looking for 'unikernel' tools, a few already exist.

I hadn’t heard this term before so that will help my search.

MirageOS is not Rust, but in the ballpark!

https://mirage.io/

I am also interested in OCaml, so thanks for sharing this. I’ve heard it’s higher level than Rust and has GC. I’ve spent most of my career using Java and after a few weeks of Rust I love many things about it but feel managing lifecycles and ownership might be too much for me.

Hang in there. Lifecycles manage themselves, mostly. You can write large, complete applications without handling a single explicit lifetime. The borrow checker becomes second nature once you get the hang of it. Don't feel bad for using Rc/Refcell/Clone everywhere. Your code will still be faster than it would have been in most other languages.

The thing about GC is that it doesn't free you from having to think about lifecycle management, it just frees you from having to write it down. I've seen a few memory leaks in Java programs due to people not putting enough thought into when a piece of data is no longer needed.

Granted, most programs don't run long enough/process enough data for poor lifecycle hygiene to become noticeable in GC'd languages.

I have also found OSv to be interesting.

https://osv.io/

Unikernel is kind of the opposite of microkernel, though; a unikernel merges the application "upwards" into the kernel, while a microkernel system tries to split responsibility into lots of not-especially-trusted pieces.

At the same time, a unikernel will usually rely on a unified upstream virtual machine, which a hypervisor will provide, because you're rarely going to run unikernel applications on exclusive bare metal.

In that context you can think of the hypervisor as your microkernel which provides a unified but very low level API to the unikernel-based microservices.

Yes. Whereas in a traditional setup you'd have an "OS" running an "executable", the hypervisor-unikernel setup the unikernel plays the role of the executable. This indicates that something is very wrong with the OS APIs that this is considered the best way to arrange the desired isolation level!

When we add WASM support to https://github.com/auxoncorp/ferros it'll sorta be like what you're angling at there in your description.

Thank you

> 1. The purpose of the OS would be to run microservices only

Ah, a microkernel approach! It's not a bad idea but tends to lose out in performance terms.

> Some interim solution for providing sockets would have to be built until wasi spec supports it

With a little ambition, and an inter-process communication framework, you could have a "network card microservice" that's got access to the PCIe registers and takes packets in and out. You could then either do "user-mode networking" in the application or run a normal IP stack to hand out packets to more normal looking sockets.

> No file system, no writing to disc

No reason why you can't write a NVMe driver in WASM...

Xous, the OS that runs on the precursor, may be of interest to look at too:

https://github.com/betrusted-io/xous-core

It is written in Rust and is targeted for a RISC-V

RedoxOS is the biggest and designed for desktop.

Xous for devices.

There is also Hubris and TockOS for embedded and IT.

There are a few others.

Was curious to understand the original Japanese version of this line, emphasis added to the relevant portion:

> it's a clean, modern RISC architecture without all the legacy crud of MIPS

Particularly the word "crud", and its implied denigration. Indeed, amusingly, the original Japanese translates:

> but I think it is a sophisticated RISC ISA with some negative legacies gone

What's funny is that one man 'crud' is another regret: MIPS had arithmetic operations which detected integer overflow..

I took an OS class but wonder nonetheless: If you aim for 0 compatibility, perhaps even can design your cpu, could an OS be much simpler? Could some standard components (paging etc.) be avoided altogether? Or is what we have a minimal set already?

> If you aim for 0 compatibility, perhaps even can design your cpu, could an OS be much simpler?

Yes, but a lot (possibly most) of the benefit is likely to be from dropping bug-for-bug compatibility with defects of preexisting operating systems and instruction set architectures individually. (Like x86 segmentation or unix tcsetattr/termios, for a pair of very obvious examples.)

For paging specifically, you'll have something functionally equivalent unless your system is deficient in the same ways a 6502 is (no virtual memory or memory protection). You might come up with something functionally equivalent but better, but if so that would be a novel discovery (either in the scientific sense, or in the sense of "I discovered a research paper from 1960 that solves this problem trivially as long as you have larger than 6-bit bytes and more than 64K[0] of RAM.").

0: Something like four rooms of vacuum tubes in 1960 money, which is why it never caught on. (/not-even-all-that-s)

The last part of this comment is brilliant. There may be large swathes of outstanding old research that had requirements that back then were out of reach but are now entirely mundane and easily achievable (even if they still wouldn't have a hope of reaching mainstream because of compatibility).

Yeah, you can make an OS incredibly simply. Virtual memory is usually supported in hardware, but files, forking, IPC aren't necessary at all to make a functional OS. The most minimal useful system you could get would probably be something like a bare-metal forth.

Check out some microkernel systems (eg. Hurd). They have their problems, but the core OS is very minimal.

The microkernel idea is to move everything that can be moved, out to user-space. The minimal OS services is typically thread management, address space management, and some kind of IPC. And if you choose not to provide virtual memory or memory protection, you have a very tiny core OS.

Everything else, all the "normal" OS services can be provided by application level programs - device handlers, file systems, network stacks, time services, graphics,... almost everything.

Minimal set to do what, though?

IMO a lot of the complexity of what an OS comes from:

* the illusion of multi-tasking! (context switching) (when you have one CPU-core)

* the bridge to your hardware! (e.g. ignoring the basics like your CPU/RAM/motherboard/IO devices, network interfaces themselves are an endless, unreliable torrent of data!)

* security! (unless you are literally God and make everything yourself, you have to trust ~something~ external)

* and more....

and complexity for CPUs (even ignoring legacy cruft) comes from :

* Branch prediction

* Register renaming

* Optimizing dat IPC

* More hacks for performance!

* Performance!!!

* and more....

Because at the end of the day, no matter how "simple" things are, creating the universe from scratch is never simple

Please please please, all new OSes need to batch all syscalls by default, and all security contexts need to take into account all of program-origin, program, and user. Also please limit whatever your equivalent of root to deny and delete. Thanks

Hi, just out of interest since I am a novice in the area, I’d appreciate if you could elaborate on this comment.

1. Batching syscalls (and to me that implies copy-exactly-once IO, AKA Zero Copy in Linux/rust) means you can better manage the cost of performing syscalls and IO in high throughput and low latency use case. Eg if you expect a bunch of network packets, the physical card, driver, network stack should split packet headers and data in distinct buffers, each contiguous, in memory. With batched syscalls, you could also instruct the kernel to memmap a file in memory, and finally combine both syscalls into a single copy from Io to Io giving the memmap buffer as the output buffer to the network stack. I don't know how this could be done today, even with iouring, but I expect this would significantly outperform existing solutions as there would be a single copy operation instead of at least 3.

2. Per origin, per program, and per identity security context I think is required to deal away with the current prerequisite of all web browsers that the underlying system be uncompromised. Basically a world where every js bundle gets executed with it's own user as its own process and having to explicitly request access to your data.

3. Combined with the above to limit the risk of compromised root accounts, if they are limited to causing DOS and data loss it's much less dangerous than a world where your entire life can be usurped by assholes with a 0day. This implies major changes in driver architectures of OS/kernels but I think it's entirely unreasonable not to make these changes. The world has changed since the 90s.

My phone has this thing where apps ask permission for my contacts. I would prefer something that worked more like, App says to OS, please let the user select a contact and then give me an opaque identifier that I can use to send data to the contact. This way the desired functionality is there, but the app never gets the contact list.

This is what Flatpak tries to do; provide "portal" APIs, that only provide a small slice of your system (one file, one folder, one screen to share), instead of allowing rampart access to the entire system and hoping that the program does nothing wrong.

I also wish more Flatpak applications actually used those sandboxing options.

> batch all syscalls by default

Switching from user mode to kernel mode and back (a “context switch”) is expensive. Traditionally, OSes did that on every system call. You can go faster if you have some mechanism for sending more than one system call in a single context switch.

> security context

Basically, what is a program allowed to do? Traditionally, Unixy (nowadays, Linux and friends) limit this based on just the current user, assuming that the user trusts every program they run. But nowadays you often don’t trust every program you run, and want it to be in its own limited sandbox.

> limit […] root

Most OSes have a super user that can do anything (pid 0 on Unixy things; Windows is more complex but still has them). They’re saying to not have that, instead make the super user only capable of stopping problematic things, not creating new things.

This is the one I’m more ambivalent on. At some point something has to set up the whole system, and the thing that kicks that off is going to look pretty superuser-like to me.

Absolutely batch and use asynchronous completion. All modern systems (NFSv4, GPUs, NVMe, etc) uses a variation of the theme. However, one should also work on lowering the context switch cost.

Another notion that I find interesting is one-address space. You still get per process protection, but address space is global. Supporting virtual memory is extremely expensive (we are paying the price today with hardware table walkers, multi-level caching of translations, various flushing on context switches). It is much cheaper if we only have to implement permissions (there are many options here) and it can make zero-copy process communication much cheaper.

Also, if you are giving up on paging, we can move beyond the 4096 byte page which we got with the 1962 Atlas (it had the equivalent of 96 KiB total memory; if pages had kept that ratio we would be using ~ 8 GiB pages today).

> Another notion that I find interesting is one-address space. ... Supporting virtual memory is extremely expensive

Would it be possible to make linux work with one address space? It seems like that should be a reasonably easy thing to do given how many different hardware platforms linux runs on. And it'd be interesting to know how much performance uplift you get from not flushing as much during context switches.

Or are there more complex interactions with different parts of the kernel that I'm missing?

I didn’t assume Linux binaries would work and I don’t see how to make that work, but general application that doesn’t depend on mmap or the Linux ABI can work.

> Would it be possible to make linux work with one address space?

Not without major changes to programs' ABI. On Linux with an address space per process, programs depend on having their private resources on fixed addresses.

The upcoming/vaporware The Mill CPU offers only a single address space, and emulates fixed addresses by aliasing a fixed part of each process' address space to somewhere else — in hardware.

In software, the ABI would have to pass a pointer to each callee's context when calling them. This is e.g. what you did in AmigaOS when you called a library function. But not all functions can be this way. You don't want all "function pointers" to be fat: function and context. For those to work, dereferenced functions would need to either belong to the program binary only, be "pure" (not access any global variables) or use a system service (on a fixed address..) to look up its context.

But the benefit of virtual memory is the contiguous memory (that those virtual indirections allows for) - ignoring virtual memory we risk falling down again to the "you need 600kb [of contiguous] conventional memory" issues of fragmented memory.

A MMU is complex, but allows simpler programs in the entire system overall.

I agree that abandoning virtual memory entirely brings back old problems, but there is a middle ground of a single virtual address space. This limits flushes to page table changes and makes it easier to move the MMU out of the CPU cores/caches. Page table size issues can be mitigated by larger or mixed size pages (and already is in x86_64/ARM)

> Also please limit whatever your equivalent of root to deny and delete.

So what creates/specifies user contexts, loads drivers, and installs kernel updates?

After a look at RISC-V's privileged spec, I doubt I'd pick an ISA other than RISC-V to target, if I felt like writing a new OS.

Beautiful and simple, as everything else RISC-V.

What's the simplest way to get this running on hardware without emulation, assuming that one has no RISC-V hardware at all?

Either port to a processor you have or build a RISC-V core yourself, say, with an FPGA. There is an abundance of designs you can use, but actually making a simple core that can support this isn't hard if you aren't too ambitious with the performance.

(I know of at least one RISC-V core implemented with 74-series TTL chips and a vacuum-tube implementation is surely in the works somewhere).

is there any updated xv6-rust (prefer X64, but at least X86)?

I already tried:

- https://github.com/connorkuehl/xv6-rust - https://github.com/tiqwab/xv6-rust

Cannot build both succesfully with latest Rust on Mac. Maybe I need to use Linux for this purpose...

I wonder if old BeOS source code can be opensourced. it would be a great alternative to port on risc-v from the getgo.

Do you know haiku-os.org? Some people did this but the long way round...

Writing an OS in Rust to run on RISC-V in a blockchain-based flying car

How long before we create a meta language that can then be auto-translated to any number of languages — Rust included - before final compilation?

(Or has this been tried 33 times and always failed?)

> How long before we create a meta language that can then be auto-translated to any number of languages — Rust included - before final compilation?

What reason would someone have for doing such a thing?

so a language that has the constraints of every single programming language at once? (at least if your goal is to have the output be halfway legible/idiomatic)

Something like Haxe? https://haxe.org/

Obviously not “any” language but it has more compile targets than your average bear.

That's what Rust, C, and many other languages are. "Meta languages" that ultimately compile into any number of machine language dialects.

This is arguably already the state of things.

Rust might get compiled down through MIR, down through LLVM IR, down to assembly or wasm... which then might be JIT or AOT (re)compiled into other bytecodes... which might perhaps be decompiled back up to C... and C might be retranslated back to horrific unsafe-spamming Rust by the likes of https://c2rust.com/. We've come full circle!

The main issue is that retranslating high level languages into other high level languages isn't something that there's actually a lot of demand for, especially commercially, especially given the N x M translation matrix going on. So a lot of the projects "stabilize" (get abandoned). And automatically translating between the idioms of those languages gets even nastier in terms of matrix bloat.

Well, you've got stuff like MSIL and JVM bytecodes which are higher level, and preserve more type information, and can be compiled to / decompiled from while still preserving more structure, but they still form competing incompatible ecosystems.

The very premise is bad. You want to use Rust precisely so that you can write Rust code. The selling point of Rust and most other languages is that they provide a sane, safe, convenient, easy... (pick some of them) interface to the programmer.

The Unity game engine created a library that transpiles C#(technically the byte-code) into C++[1].

The generated code isn’t particularly readable and it comes with some security issues[2] but the upshot is you have the benefits of a C++ compiler.

[1]https://docs.unity3d.com/530/Documentation/Manual/IL2CPP.htm...

[2] https://github.com/djkaty/Il2CppInspector

> the benefits of a C++ compiler.

Is there any? I think most modern compiler share the same codegen backend.

You don't get the benefit of the frontend when you transpile.

> Is there any? I think most modern compiler share the same codegen backend.

JIT-focused .NET is one of the ecosystems disparate from AOT-focused LLVM. While there's a bit more cross pollination now, at the time the C# to C++ transpiler was authored, there was much less so. I've taken a stab at porting mono to a new platform at a similar time period - it was rather nontrivial (I ran out of time and thus failed.) Worse still, many of Unity's targets (e.g. iOS, XB1, ...) explicitly ban JIT technology in the name of security, which wasn't something I had to deal with.

A MSIL bytecode -> C++ translator might be pretty quick and dirty... yet effective, gives you AOT compilation, avoids the need to explicitly target every architecture by hand. For all it's faults, it's not too terribly hard to figure out how to compile C++ for a given platform, typically, generally requiring exactly zero reverse engineering.

what I need is a language the can directly call c and cross language compile with c so I don't have to write c! especially that nasty pre processor. cause there so much good c libraries that I want to consume and they're only available in c!!!

Zig. https://ziglearn.org/chapter-4/

I use Julia for this. Its ccall facility is very handy for C FFI calls, and Julia itself is very fast and performant with just a bit of type annotation.

https://docs.julialang.org/en/v1/manual/calling-c-and-fortra...

julia is so nice! can’t believe I didn’t try this before! thank you very much

Calling C from rust is easy enough

LLVM languages can all natively link with C code, since any LLVM bytecode can directly call other LLVM bytecode. So Zig, C++, Obj-C, Swift and Rust can all be compiled together.

But C compatibility is everywhere. Almost every language has a FFI (foreign function interface) for C code. C interop shows up in every language because at the end of the day every language needs to be able to talk to the operating system. For example, to read and write files your language needs to call functions in libc (or equivalent). And that library is written in C.

So, Nodejs can call C via has native modules & NAPI. Java can call C through JNI. Ruby has the ffi gem. Python, Luajit, Go, C#, ... the list goes on. They all have a mechanism to call C code.

If you're looking for another language that can live alongside your C code, you can choose any.

thank you so much! I didnt know that about LLVM. and I love Swift!!!

However, one typically needs to write out a description of the “C” interface. And if the “C” header files are large it can be a lot of work. For example, I’m unaware of any complete Python bindings to the win32api, user32, kernel32, gdi32.

Can we use NAPI and JNI together to bridge nodejs and Java/Android

Can we use NAPI and JNI together to bridge Node and Java/Android applications

Then you really want Nim.

Eiffel initially compiled to C, then used the machine's C compiler to compile to native code. Later it would support outputting to java bytecode and .net CIL; shouldn't be any reason not to have other output formats.

So the main question is... why?

The closest you get to this sort of thing in practice is something like a parser generator, or an automatic interface generator. ANTLR and Tree-sitter both allow generating parsers in a variety of languages, but their input is basically a description of an AST in a highly specialized and extremely declarative context.

Once you start having actual logic in your code generation, there is very little benefit to codegening to a high-level language instead of going directly to a compiler IR, a bytecode, or assembly directly. The places where you see that happening--JavaScript being the biggest one--is largely limited to where there is no other alternative.

Nim compiles to C, C++, Objective-C, and Javascript.

Technically it'd probably be possible to compile down to Rust. It's gc is essentially wrapping in Rc's. Though not sure it'd bring much vs linking to Nim's C output.

C++ and Objective C started out as an extra layer compiling to C

(2019) ?

Reply on news.ycombinator.com