Gokrazy: A pure-Go userland for Raspberry Pi 3 appliances
gokrazy.org
gokrazy.org
(The gokrazy and u-root people all work at Google, I just contacted them to see if we can help each other. Not a competing project.)
There are a few Basic, Pascal, Oberon and Java (not Android related) options available, but only a minority cares for them.
Still, big enough to keep those vendors doing business.
Having a minimal bare metal runtime for Go would also be nice, but it is a gigantic effort to make it portable across all types of hardware.
gokrazy generates strong passwords by default and listens only on private IP addresses by default to reduce the attack surface.
Nerves is based on a Buildroot core but most of the system is built with Elixir and BEAM/OTP. It's fantastic having a high level language built for robustness and reasonable security. Still, I have dreams of porting BEAM to Redox OS one day [2] and replacing the C userland-bits with Rust-bits. Especially if you could represent the micro-kernel services as actors. That would be amazing!
1: https://nerves-project.org/ 2: https://github.com/redox-os/redox
Google has created a stack called NERF (Non-Extensible Reduced Firmware) for this:
https://firmwaresecurity.com/2017/07/23/google-nerf-non-exte...
There's a presentation about it on youtube:
Want something like this for one of the languages that I know... Node.js, Python or Rust. I'd include C++11, though for this kind of deployment method I think one needs packages easily available in a way separate from Linux distro (since there is none).
Instead, a filesystem like btrfs should be used which can give the appearance of multiple versions, yet only use actual storage space where versions differ.
I'd prefer the root filesystem to be something using a FUSE-like layer which can store format specific binary diff/patches between versions, and then transparently apply (and cache) them on access.
Taking it a step further, I wish desktop linux distros had a package-manager-fs, which is a readonly virtual filesystem which displayed all the files installed by all packages, but the underlying real filesystem only contained all the original compressed packages and a cache of recently used files.
But it is very simple and robust. What one tests pre rollout is identical to what will go on production devices, down to the byte. Each device with same version is identical. It can be cryptographically verified easily. Automatic rollback can be performed on update fail, including self-checks.
This approach of shipping a custom userland without any of the generic Linux userland should drastically bring down the size.
A minimal gokrazy root file system is 16MB in size. On a 8GB SD card, I certainly am willing to sacrifice another 16MB to get A/B updates :).
The initial installation of the OS is simply done by unzipping a carefully crafted [1] install.zip file to the Raspberry Pi SD card. By exploiting zsync features, it's quite efficient to reconstruct the original install.zip from the unpacked files. Once that's done, all future updates also use zsync and only fetch changes to the locally stored 'install.zip' file. After updating this file we then unpacked everything again to the next A/B partition, so the exact state of the system is always known. Updates are quite small that way. It also helps that the complete OS is only ~37MB compressed at the moment.
[1] The root file system is squashfs and is stored uncompressed in the zip file to avoid double compression and allows zsync to use the squashfs blocksize to find a minimal set of changes. Similarly the initramfs.gz is compressed --rsyncable and also also stored uncompressed in the zip file.
This is so cool! I’m very happy to learn an effort like this exists, and looking forward to perusing the code for it... in Go (a language I really enjoy reading).
https://github.com/gokrazy/kernel/blob/3ecdc901da51c2c5b6bf7...
Apologies if this is not helpful.
"For a long time, we were unhappy with having to care about security issues and Linux distribution maintenance on our various Raspberry Pis."
But-- there are so many applications for having a full IoT Linux distribution that I can't believe this isn't already a solved problem. I'm imagining an ultra-stable distro that is easy to deploy to a headless machine, handles security updates, handles flaky wi-fi issues, tests the SD card (I've had more than a few burn out on me), and automatically logs to the cloud for easy debugging.
In other words: Why am I still maintaining my RasPis as if they are apache servers from the 1990s?
It's container-based. Works super-well on a Pi3.
Sad but true. There's still a few companies selling microkernel-based solutions in defense and mobile with Linux user-lands. They'd OEM license their stacks for a new set of products aimed to do this. They cost a fortune. Whoever pays it wont get it back, though, cuz folks dont usually pay for security in that volume. So, nobody is doing it for small players for now.
If the system only contains the Linux kernel and your software you only have to worry about your security issues (unavoidable) and Linux security issues (also difficult to avoid - too much work to write a new kernel too, plus remote Linux vulnerabilities are quite rare).
A big part of the motivation for gokrazy is to use a safe, high-level programming language.
This allows us to re-implement the userland with small amounts of code, see e.g. the NTP client at https://github.com/gokrazy/gokrazy/blob/master/cmd/ntp/ntp.g...
Replacing the implementation language eliminates whole classes of attacks, while stripping down an existing Linux distribution can only go so far. Plus, it’s not clear to me how one would maintain such a stripped-down distribution in a way that isn’t a lot of effort.
The easiest solution to me seems to be to grab the tools you need from the existing userland, instead of reimplementing them in Go. Currently you have reimplemented some of the userland, and packaged it up into a distro. So you have to maintain a stripped down distribution and the tools that you reimplemented. If you just took existing tools you would only have to do the latter. Am I missing something here?
Have a look at the git history to see how little maintenance is required in practice. I have barely had to touch the code over the last year of the project. Just incorporating new upstream versions of a typical Linux userland would have been more work.
Also note that “distribution” is a rather big word for the few files we’re talking about:
/mnt/gokrazy-root % find -type f ./cacerts ./gokr-pw.txt ./gokrazy/dhcp ./gokrazy/ntp ./gokrazy/init ./user/hello ./localtim ./hostname
If my reasoning doesn’t convince you, so be it. Not every project has to make sense to everybody :). I wanted to do a pure-Go userland, and I’m happy with the amount of work this results in and the properties that come out of this decision (easily automated updates, massively reduced attack surface).
When your computer runs headless, the init/process tree system is userland. Background (cron) jobs are userland, daemons are userland. Etc.
... that will have security issues, require updates, distribution maintenance, and all that stuff they claim is unacceptable ?
Updates are easy to fully automate with gokrazy (and I’ve been using that mechanism for a year at this point) because assembling a new image and installing it over the network is just one simple command. New kernel and firmware versions are provided automatically once they are verified to boot on real hardware.
There is no maintenance aspect in the actual installations (i.e. you don’t need to SSH into them, ran commands to update, etc.) — the root file system is read-only, and an update overwrites it completely. Unless the applications you want to use with gokrazy introduce state, gokrazy defaults to being stateless.
Hope that clarifies things, let me know if you have further questions.
Why is this choice better than a language that provides memory safety using linear types so you don't even need a runtime or garbage collector?
Of course I used my favorite programming language for building gokrazy :).
I love your stuff! I enjoyed your episode on Go Time as well!
Cheers!
I can't resist while here thanking you for i3. I've been a mac user recently, but will likely be back to linux in the next year or so. Using i3 again will be probably be the greatest pleasure involved in that return ;)
gokrazy’s unit of application is a Go package, so you’d need to write a Go program which then executes the Lua VM. I don’t know what dependencies Lua has — if it cannot be compiled statically, I’d recommend against using Lua in this context.
Not really a huge number, no. Java is probably the most obvious, if you count that as compiled. Maybe Haskell or D? I suppose even Swift of Objective C if you include RC in the definition of GC.
I would say the focus shouldn't be entirely on GC, however. Safety as a whole is the most important factor, and it's possible to be say without GC, as you point out. I reckon Go is actually quite an attractive language for this sort of application regardless – quite safe, quite lightweight, good libraries.
Also multithreading gets some improvements which will take more of the manual sync details away from the business logic. (http://www.modernescpp.com/index.php/multithreading-in-c-17-...)
It's still delivered attached to a foot-gun, but you could do worse.
On safe languages an out-of-bounds will cause a crash on the access point, while on unsafe languages it will silently corrupt memory.
If you have an out of bound in C, your program keeps running in 9/10 cases. In Go you'll cause a panic in 10/10 cases which you can either handle and propagate as error or let the program crash entirely, the memory will be in a defined state if you choose to recover (your array might be garbage but you won't have written data into random memory belonging to other threads)
The %p formatting used to gain access to the pointer of a variable uses the unsafe package, the formatting itself is incredibly rarely use to begin with.
I don't see anybody doing any security-related exploits doing that unless you explicitly write for it.
Go won't do out of bounds, edges cases do happen. You can find the same stuff on Java and Rust (yes, I had OoB in Rust, though on a baremetal environment without OS, even Rust ain't safe.)
> You'll notice my address() trick to get the address of a function or variable that I used in my previous exploit. We're relying on package fmt and its %p format specifier which is implemented by using the unsafe package, so we don't need to use it ourselves.
The unsafe address() is used to get the address of a function to store in the int, to demonstrate the potential for shellcode injection. But you could just as easily assign pp := 12345 and produce a SIGSEGV or SIGILL, which would not be possible if Go were memory safe.
The exploit was only possible thanks to using the unsafe package, even if indirectly.
It is exactly because Go is memory safe that a SIGSEGV or SIGILL is produced the exact moment the memory gets corrupted and not 2 hours later in a complete different stack trace.
> A crash on out of bounds will always happen, unless unsafe package is being used to subvert memory accesses, or if you've used something that exposes details that only the unsafe package exposes, such as fmt's %p specifier.
I mean, you are like, the ultimate pedant, so I figured you'd uniquely appreciate this.
You can't just say, "use unsafe or anything that uses unsafe," because it's turtles all the way down. You've got to draw that line somewhere. If the %p specifier really is required for an exploit and there's nothing else that exposes that (sans unsafe), then that's great news! But is that the whole story?
I really like Go, and suspect a Go app would beat the pants off anything powered by Electron (performance wise).
In case any of periph.io’s features require changes to the kernel configuration, I’m happy to change it accordingly.