looking forward to the unwritten details / future posts too, particularly:
- How to handle network and IP duplicates on cloned VMs
and
- Turning a Dockerfile into a rootfs for the MicroVM (quickly)
I'll make sure we write about the other topics as well. For the network, we run the VM in its own network namespace on the host, and we give every VM the same IP. We then use an iptable rule to rewrite every incoming and outgoing packet to the IP that the host has assigned for the VM.
Another use case I was thinking of was stateful compilers like scala where warming up the compiler is expensive, often a CI task too.
Disclaimer: I’m the author.
- you can dump the image using `docker save <name>`. - you can then get a list of the tarballs in this image by extracting this tarball and reading the file `manifest.json`; `Config` -> `Layers` will give you a list of tarballs (see undocker for how to do this: https://github.com/larsks/undocker) - Untar these in a directory and use linux tools to convert this dir to a rootfs.
Unrelated TIL: AWS Fargate has supported Windows since last October. I work at AWS and “specialize” in serverless and I didn’t know that.
On the other hand, CodeBuild has supported Windows containers for years and at least CodeBuild for Linux is based on Fargate, so the service team figured something out. (I had to figure out how to word that. I can’t say “they figured it out” since I work for the same company. But I couldn’t say “we” since I’m so far removed from any service team in the consulting department that it would be disingenuous)
Another (unrelated) test we've done is on overprovisioning memory. We were able to run 200 VMs (all running Vite dev server where a file was changed every second) with 2GB RAM per VM, on a node with 128GB RAM. Because we were mapping the memory files on disk directly to the VM, the VM would automatically "swap" the memory back to the memory file when it had memory pressure. The bottleneck here was CPU.
The microVM emulates the minimal possible set of devices needed to run, such as disks and network devices, and in the specific case of firecracker, through the use of the virtio model. So it can theoretically use huge amounts of memory of a large vCPU count and still be a microvm.