Chrome OS KVM - A component written in Rust
chromium.googlesource.com
chromium.googlesource.com
Aside: I really like what the ChromeOS team has done over the years to advance the state of OS security for consumers, keep up the good work!
> // This is safe; nothing else will use or hold onto the raw sock fd.
> Ok(unsafe { net::UdpSocket::from_raw_fd(sock) })
https://chromium.googlesource.com/chromiumos/platform/crosvm...
The important bit to me is that the default pointer type and the standard library don't start you out on the wrong foot.
The "this is safe" comment is just a (very verbose) explanation as to why the wrapper code is doing the correct thing. Notably, there's another similar comment just above it. In fact, looking through the file if anything I'm quite impressed at how carefully it's written.
(When I've pondered this question before, "document every instance of `unsafe`" has been one of my policies (as obvious as it may seem), so I think you're on the right track!)
// This is safe since we check the return value.
let sock = unsafe { libc::socket(libc::AF_INET, libc::SOCK_DGRAM, 0) };
if sock < 0 {
return Err(Error::CreateSocket(IoError::last_os_error()));
}
Just my opinion.
For one, there is the current problems of documentation - it's not documented what features/invariants the optimizer and language actually require to be true, and the nitty gritty details are very fuzzy, so switching from `unsafe` to `safe` is error prone and you're going to get it wrong. The more you do it, the more likely it is your code will be broken in the future when you find out something you thought was OK isn't actually something the Rust devs like or isn't something they wanted you doing. If you do more of the `unsafe` work in one big `unsafe` block rather then jumping in and out, you're less likely to have issues in the future because there's less points where you have to ensure all the Rust invariants are met.
But the bigger detail for me is that, even if the above problem is fixed, `unsafe` doesn't really denote the areas we would consider the `unsafe` areas anyway, so "getting out of there ASAP" is not always a helpful mindset and can easily be counter-productive and result in you marking things `safe` when they're not actually `safe`. For example, dereferencing a pointer is `unsafe`, but doing pointer-arithmetic is `safe`. So you can easily just wrap the dereference in an `unsafe` block and your technically good to go (You can even wrap it in a pretty interface, like I've seen people do). But all the spots where you do pointer-arithmetic can easily introduce bugs into your `unsafe` code, making it hardly any better than C code that could have the same problem (Half the point of using Rust is to avoid bugs from unchecked pointer-arithmetic!).
My point being, just because your `unsafe` blocks are small doesn't tell you anything about the correctness of them, and it likely means they rely on outside information to be correct. And if that is the case, then that outside code is effectively just as dangerous as your `unsafe` code. This may be obvious to you, and I apologize if it is, but this is an issue/misconception I see a lot. IMO, you should mark anything `unsafe` if using it within the bounds of `safe` Rust could potentially cause `unsafe` code to fail, even if the code itself is completely `safe` code. Only if you have an interface the meets all the invariants that Rust requires should you allow it to be considered `safe`.
Rust's safety guarantees are mostly there to prevent you from making a certain class of coding errors. "unsafe" means that you've disabled those protections, but that just means that you have to be extra careful not to make those mistakes, and that people reviewing your code need to spend extra time to make sure that you haven't made those mistakes.
In the current state, much of that work is left to LLVM which is designed more around the C/C++ memory model and only some of the ownership and aliasing information can be represented in that form.
I am not arguing that Rust is faster now, only that it can be without fundamentally changing the language.
A couple of points here. Rust is generally as fast as C and C++, but on top of that it is memory and data race safe. Said another way, there is a safe language alternative which doesn’t have the pitfalls of C/C++, and that’s a great thing!
That being said, there was a ton of work getting Rust to play nice within the Chrome OS build environment. The Rust folks have been super helpful in answering my questions though.
I ran into a similar use case in one of my own projects—a vobsub subtitle decoder, which parses complicated binary data, and which I someday want to run as web service. So obviously, I want to ensure that there are no vulnerabilities in my code.
I wrote the code in Rust, and then I used 'cargo fuzz' to try and find vulnerabilities. After running a billion(!) fuzz iterations, I found 5 bugs (see the 'vobsub' section of the trophy case for a list https://github.com/rust-fuzz/trophy-case).
Happily, not one of those bugs could actually be escalated into an actual exploit. In each case, Rust's various runtime checks successfully caught the problem and turned it into a controlled panic. (In practice, this would restart the web server cleanly.)
So my takeaway from this was that whenever I want a language (1) with no GC, but (2) which I can trust in a security-critical context, Rust is an excellent choice. The fact that I can statically link Linux binaries (like with Go) is a nice plus.
This has been more or less our experience with fuzzing rust code in firefox too, fwiw. Fuzzing found a lot of panics (and debug assertions / "safe" overflow assertions). In one case it actually found a bug that had been under the radar in the analogous Gecko code for around a decade.
Also, the Wayland stuff looks cool but I'm not sure how you are managing the buffers with just the wl protocol.
There is a Wayland crate for Rust that also has support for generating protocol from the xml descriptions, not sure if you used that there.
The library is here: https://github.com/Smithay/wayland-rs and the part you would want is the `wayland-scanner` crate in that repo.
This seems like a good follow on to the work done in Go and python a while back at Google, though it would be cool to support the virtio p9fs as a root filesystem.
And, I know you can't say anything about this, but I'm happy to see the arm support in there, maybe it's possible that Google supports a fully virtualized Android device with the ability to run first class Linux, ChromeOS, etc. even on locked bootloader devices.
It would even be possible to keep a tiny resident e911-compliant dialer persistant as part of a lock screen.
Anyway, awesome work, I might have to revive kvmd at some point.
Sadly, not many of the components within crosvm would make good crates in crates.io. The crates in crosvm tend to be laser focused on solving a specific use case within crosvm. There are more general versions of lots of the functionality we have in crosvm that exist in crates.io (e.g. eventfd or memory maps) that we skipped to avoid excessive external dependencies.
>Also, the Wayland stuff looks cool but I'm not sure how you are managing the buffers with just the wl protocol.
The virtio wayland I designed is intended to be somewhat agnostic of the underlying wayland protocol for simplicity. It just passes along the protocol bytes to the host's wayland compositor. In order to share FDs with the host (to support e.g. buffers and keymaps), crosvm has a mapping of virtual file descriptor IDs known to the guest kernel to host FDs that get passed along to the wayland compositor.
>though it would be cool to support the virtio p9fs as a root filesystem.
We considered it, but for our application, 9pfs was not going to be optimal. That being said, we'd welcome patches that added support for it. :)
>And, I know you can't say anything about this, but I'm happy to see the arm support in there
The ARM support is rather preliminary. crosvm will compile for ARM but it has yet to succesfully boot a VM.
I was actually referring to your kvm and kvm-sys crates, through it seems that some of the memory management stuff is closely tied to that. I ask because I tried to build a kvmd daemon using the kvm-rs crate that's out there, but ran into some issues. At the time bindgen was not quite up to the task of handling the kernel headers itself as well.
> The virtio wayland I designed is intended to be somewhat agnostic of the underlying wayland protocol for simplicity. It just passes along the protocol bytes to the host's wayland compositor. In order to share FDs with the host (to support e.g. buffers and keymaps), crosvm has a mapping of virtual file descriptor IDs known to the guest kernel to host FDs that get passed along to the wayland compositor.
Okay, I guess I'm more asking how software running on the VMs are accessing buffers on the GPU which are managed by the Wayland server, as I don't see anything in the virtio folders referring to this. I guess this is just using some kind of shared memory region then? I'll have to look at it more later.
Also, this low-level Wayland really seems like it could be a standard way to access GPUs through a hypervisor, maybe something that could be implemented by (k)qemu or others.
> We considered it, but for our application, 9pfs was not going to be optimal. That being said, we'd welcome patches that added support for it. :)
Yes, and I might try to get to that sometime. I really like the approach starting with vmlinux and not a BIOS implementation, anything to make the OS boot faster in the VM is good.
> The ARM support is rather preliminary. crosvm will compile for ARM but it has yet to succesfully boot a VM.
Understood.
I also find your Rust style extremely readable. Great job!
But we also don't get in the way of teams trying to experiment and see what will work for them. (otherwise, we'd never be able to know what languages to sanction)
https://google.github.io/styleguide/
All the languages you mention are there.
If the Pixelbook can run Window or Linux in a VM, then its price slides a little closer towards justifiable.
This may be used as part of running Android apps on ChromeOS in more secure sandboxes, but the Wayland integration suggests to me this might be for running traditional Linux desktop applications on ChromeOS.
I think there's a good chance we'll find out something at the rumored upcoming Pixelbook launch.
Go still makes network code and certain models of concurrency stupidly simple.
Rust is more of a replacement for C/C++ for me.
I'm on the lookout for channels and green-threads in Rust (so I can basically write borrow-checked Go-style code in Rust).
Rust is getting an unstable form of async and await as macros/syntax extensions, and there are RFCs discussing adding them to the language in some form. This would still be a wrapper for futures, but a more ergonomic way of using them.
I didn’t understand you to be honest.
Problem with your idea, is that low level kernel will use a lot of unsafe Rust, which will lose lot of benefits.
I've actually worked on a toy kernel in Rust (using the excellent tutorial at https://os.phil-opp.com/), and it turns out that, yes, you obviously need to use unsafe code to talk to the actual hardware. But in most cases, you can encapsulate the low-level hardware inside a safe API:
https://github.com/emk/toyos-rs/blob/fdc5fb8cc8152a63d1b6c85...
In this example, only I/O port creation is an unsafe API, because you need to specify a memory address to read and write. But once the port is created (pointed at an appropriate address!), it's perfectly safe to use.
So, yes, kernel-space Rust will use "unsafe" far more often than regular Rust code. But you can still make at least 80% of your code safe, and maybe much more. And the remaining "unsafe" APIs act as a useful warning to pay attention to what you're doing. Plus, Rust is a really nice language to write kernel code in, anyway.
"Given enough eyeballs, all bugs are shallow" - but it helps when the eyeballs are focussed! :-)
I'm not a professional RE so my experience is limited, but when I went looking for vulns that's how I went about it, and I think that's generally the case.
With rust there's significantly less guesswork. That parser doesn't use unsafe? OK, let's start elsewhere. That seemingly innocent code uses unsafe? Great, check that out.
You can grep for vulnerabilities, basically.
I'm probably missing something obvious. But isn't that true for most languages?
My experience is that huge amount of C code running on my computer had exactly zero issues like this today. It has been working perfectly fine.
I run GNOME which uses a paradigm of checking all inputs to a function on entry via g_return(_val)_if_fail(assertion_expression). It helps the programmer do the right thing when it comes to using APIs that are disallowing NULL input.
Two days ago, the code on my workstation hit one of those "assertions":
Sep 26 23:40:31 core transmission-gt[1084]: g_file_test: assertion 'filename != NULL' failed
No oops in the kernel recently.
Millions of lines of desktop and kernel code, and one failed assertion in two days for using NULL incorrectly in the API.
So unless your standard is absolute perfection, it works fine as is.
I don't think I've ever run into a null reference in the real world. I'm sure it happens, especially if people write "&*some_function_that_might_return_null()". But it shouldn't be a normal thing. There are lots of other issues with C++, but this has never been a major one in my experience.
They are!
And while they are not a normal thing, they are a thing and I've run in to them a handful of times in the real world, almost always the result of someone not checking for null before dereferencing a pointer.
Rust does not really have this issue.
Because I can assure you, unsafe blocks in enterprise Rust will be reviewed as much as C and C++ code currently are in most Big Corps™.
We do have such problems with native libraries killing Java and .NET processes, with unsafe being the FFI boundary.
std::vector<int> x;
x.push_back(4);
int& val = x.back();
x.push_back(5); // ahh, val may be garbage now.
Foo f;
Foo& fRef = f;
Foo g = std::move(f);My point was similar C/C++ don't have a safe subset.
If the platform has an MMU, if you've properly set up your page table mappings, etc. We are talking about writing an OS here!
Not to mention all the UB around null pointers that can cause miscompilation. Well, not technically miscompilation, but stuff like https://news.ycombinator.com/item?id=15324414
If there were a list "falsehoods software engineers believe about memory safety" this should be there.
Theoretically it's just all the same, but in practice having reduced interfaces does wonders to reduce the complexity and being able to reason about what brings problems, and what doesn't.
unsafe blocks are just a compromise where the programmer only has to prove safety of small blocks of code which aren't easily expressed in the type system.
With Rust, it's easier just to replace the C library! In particular, cross-compiling with a static musl-libc has been supported out of the box with for a while now. For pure Rust programs (or ones with small amounts of C handled by cargo), just write:
cargo build --target=x86_64-unknown-linux-musl
For programs which require external C libraries, it's a bit trickier, because you need a static version of those libraries built against musl-libc. For common libraries like OpenSSL and libpq, I have a Docker container that makes this easier: https://github.com/emk/rust-musl-builder
http://programatica.cs.pdx.edu/House/
https://llvm.org/pubs/2009-08-12-UsenixSecurity-SafeSVAOS.ht...
So, that's not really a limitation. Worst-case scenario is proving those primitives correct with external tools whose preconditions and invariants are just checked by memory-safe code calling it. The above work shows worst case might not happen, though.
Uh, this is rambling a bit. My point is that building zircon as a microkernel makes it much easier to write correct code, Rust or not. And that maybe the overhead of a rewrite wouldn't be as beneficial for the microkernel as it would be for other parts of the operating system.
I think one of the clearest examples would be drivers -- they might be properly sandboxed by a good microkernel, but they are notoriously buggy/crashy/incorrect. If I remember correctly one of the Fuchsia team members told me that they were going to support drivers written in Rust, but I could be completely wrong.
Filesystems, the network stack, whatever horrifying systemd equivalent Fuchsia grows, etc. are all examples of things I would advocate for writing in Rust before focusing on the kernel. All of these things are still security/reliability sensitive, all still terrible by Rustacean safety standards in most OSes, and conveniently don't have to be built directly as part of the kernel when you're doing a microkernel.
Just spitballing, of course.
https://github.com/fuchsia-mirror?language=go
https://groups.google.com/d/msg/golang-dev/2xuYHcP0Fdc/tKb1P...
Regarding your specific latency issue with a GC systems language, here is a Mesa/Cedar example about a network file server.
The workflow for seL4 starts with prototyping in Haskell, formalizing everything in Isabelle, and then translating into C. This results in highly unusual C code and much of it would be better serviced with a DSL. I'm guessing that C chosen because it has formally verified semantics and compilers as well as integration with proofing assistants.