Show HN: Ephemeral VMs in 1 Microsecond
github.com
github.com
I did a pass of the codebase and it seems they’re just forking processes?
It’s unclear to me where the safety guarantees come from (compared to using e.g. KVM).
Edit: it appears the safety guarantees come from libriscv[0]. As far as I can tell, these sandboxes are essentially RISC-V programs running in an isolated context (“machine”) where all the Linux syscalls are emulated and thus “safe.” Still curious what potential attack vectors may exist?
[0] https://github.com/libriscv/libriscv/tree/dfb7c85d01f01cb38f...
Staring emphermial hypervisor VMs quickly is more noteworthy (since they are often slow to start) than an emulator VM where it's expected to be fast since it's usually not much more than setting up a datastructure and executing a call to an interpreter. I clicked hoping for the former, only to find out the project is the latter.
Maybe I’m not seeing how those things are fundamentally different than “setting up a datastructure and calling an interpreter”.
A hypervisor VM typically requires more extensive setup involving hardware configuration and is usually used for running existing native code, so it often means emulating a real machine, including OS and sometimes even firmware/BIOS. There are "lightweight" environments like Firecracker, but the overhead of creating an instance is still heaver than the overhead of a function call. The instance creation overhead is high, but the instruction performance can be close to native. Microsecond VM creation is notable given the typical instance creation overhead in this case.
A emulator VM for a sandbox typically will just be a software CPU emulator with some level of OS emulation. The instance creation overhead is setting up a data structure and issuing a function call to "run" the CPU emulator. The instruction performance is generally much slower than executing native code. Microsecond VM creation is not very notable in this case.
If your running a long running process the hypervisor approach is usually superior. If your running a very short lived process (for instance a VM per http request), the emulator approach may work better.
There is also the container approach like Docker which is somewhat in between in overhead and can run at native speed on bare metal. The OS virtualization GVisor approach of capturing syscalls.
I guess what I'm trying to say is maybe the distinction you're drawing isn't really as distinct as you think it is; if this project had virtualized devices and a kernel driving them instead of passing through syscalls, would that be real virtualization, assuming we're talking about a non-native guest ISA? don't vm guest drivers abstractly just pass through to syscalls / host drivers anyway? what if there was no OS and the user's code implemented those drivers? aren't virtualized devices "just setting up a datastructure and calling a function" too? if not, what are they?
like, do you see how this is really a spectrum or collection of system components with levels of virtualization?
CPU-only virtualization with syscall sandboxes is still more secure and useful than fancy chroot.
For me the reason for the distinction between hardware virtualization and emulation based virtualization is the differences in suitability based on instance creation cost, and instruction performance, and as you mention security.
My usage was just an attempt to explain why I saw a distinction, not to pedantically define any of the terms. My apologies if you felt I used them incorrectly.
Hetzner is really cheap but not sure about the cost effectiveness for the dedicated servers. Actually I think what I saw was that I couldn't get the one I wanted in a US datacenter.
You might look into gVisor if you're running containerized workloads on a host without virt support (such as a vm without nested virtualization support).
- The VM has to be built in a safe and performant language (like Rust, Go, or - if careful - modern C++), and available as a library to integrate with the rest of the game. However I don't trust myself to write safe C/C++ (the game is being prototyped in LÖVE/Lua).
- Each VM instance needs a tight execution/instruction budget, to avoid stalling the server's main update loop; e.g. a timer/virtual "hardware interrupt", or simply counting cycles, or even something modelled after eBPF. The total number of VM instances running in a single game would also need to be limited somehow (e.g. making a key component scarce and/or non-renewable, or dividing the total instruction budget across all VMs, or requiring a player to be present in a nearby world chunk).
Use cases are something like redstone in Minecraft: curious and technically-inclined players could build contraptions, like auto-farming crops, pranks/traps, defences, fancy gates/moats, etc. Not the core of the gameplay, but rather one aspect of it, for the curious to explore, learn, have fun with.
There are many off-the-shelf VMs that do RISC-V or similar ISAs, and I'm considering picking one of those, but wondering if a RISC instruction set isn't too low-level for such a thing. On the other hand, it would be nice if the knowledge would be directly transferrable to the real world.
Anyone tried to build something similar and can share their experience?
One of the Rust-based Lua VMs, piccolo [2], was also made specifically with the ability to be interrupted in the middle of anything, by being stackless.
[1] https://docs.wasmtime.dev/api/wasmtime/struct.Store.html#met...
[2] https://news.ycombinator.com/item?id=40239029 (see "Fuel" on its article)
There are also decades of articles on how lua works with C and C++, and you can find examples for Rust and others too.
See e.g: https://www.nervos.org/knowledge-base/comparing_blockchain_v...
this is what makes it a joke
microvm’ virtual platform (microvm)
microvm is a machine type inspired by Firecracker and constructed after its machine model.
It’s a minimalist machine type without PCI nor ACPI support, designed for short-lived guests. microvm also establishes a baseline for benchmarking and optimizing both QEMU and guest operating systems, since it is optimized for both boot time and footprint.
There are a few Zach-like games which achieve this kind of thing.
The scope of the VMs they build are different, the VMs in Shenzen I/O are quite different to that of TIS-100. Likewise non-zach zach-like games like "Turing Complete" also let you build through VMs.
These aren't the kind of VMs you might expect, they're not full x86/x64 emulators, they're very simple virtual machines that have can handle a small abstracted instruction set.
If you restrict the VM to executing a limited number of instructions/cycles, then you don't need it to be super performant in terms of clock cycles.
More important is to define the memory limits of your VM. This is a key constraint that will keep processing feasible but also keeps understanding to be manageable for the player.
If I recall correctly, TIS-100 has 1 general register and 1 accumulator register in each general cell. I don't remember now if there are also special memory cells, but if there are, I suspect the total amount of memory is of the order 64 bytes or so.
Other similar VM games have more memory support, but typically top out at 16KB.
What's the actual gameplay loop you'd like to achieve? Defining that will help define and shape the constraints you put on players.
Indeed. I want something less awkward than redstone; I'm also considering something like fCPU[1] or eBPF (no jumping back = no loops = super easy to keep an upper bound on execution budget).
[1]: https://mods.factorio.com/mod/fcpu
But one of my side goals is to attract players who might be interested in contributing to the core game, so I also like whytevuhuni's suggestion[2] to embed a Lua interpreter. That's a faraway stretch goal (I still have very little in terms of prototyping), but I'd like to keep my options in mind.
[2]: https://news.ycombinator.com/item?id=42493118
> What's the actual gameplay loop you'd like to achieve? Defining that will help define and shape the constraints you put on players.
The general direction is something like Minecraft (with some Factorio / Oxygen Not Included thrown in), with focus on exploration, self-expression, collaboration, and a kind of lore/storytelling (that's deeply personal, like GRIS but still in a generative sandbox).
For example, people have made pixel art in Minecraft by meticulously placing blocks, one by one, on a huge area, and then creating an in-game map of it all[3]. I'd like my game to feature a pixel art editor instead. Same goes for in-game coding/automation: I want to reduce the friction from an idea to seeing it in-game.
[3]: https://minecraft.wiki/w/Map
Thank you for all the suggestions!
> The test program is a simple […] return string
I understand how this is required to measure the effects of sandboxing in isolation. And the result is impressive.
In what ways would you expect performance to be affected when workloads are more realistic as well?
libriscv in interpreter mode is fast compared to other interpreters, but not near native performance. As I wrote earlier in the thread using something backed by KVM is what I would do if I were architecting a solution for someone. Eg. my TinyKVM KVM-based userspace emulator would fit the bill.
Is there any cloud that provides RISC-V VM's, coupled with SQLite access for persistence?
I think when it comes to being integrated into a web server, you ideally want to use the web servers pipeline to make requests, so that you can benefit from the same logging and observability that Drogon (or other solutions) already have.
If you allow the VM to make requests on its own, you just have the IP and an open socket. Not much to debug with if (let's say) you have a customer asking about something.
> Multi-tenancy allows one server to be safely shared among many users, each of which cannot access each others or negatively affect the HTTP service.
> Specialized sandboxes are instantiated for each request and immediately destroyed after the request, all within a single microsecond.
https://www.prisma.io/blog/announcing-prisma-postgres-early-...