Firecracker is great and all, but the core idea here described works also with plain docker; i.e. there is nothing inherently firecracker specific to the basic technique
The three big differences are:
1. Docker doesn't deal with running processes (like postgres or redis), only the filesystem state
2. Docker doesn't have enough isolation, so you'd probably need to run it within qemu or firecracker for compliance in bigger teams
3. Docker-in-docker is still pretty painful, if you need to do anything nonstandard like change the size of /dev/shm, access /dev/kvm, or load kernel drivers, it'll take custom configuration.
Interesting that you're folks now use firecracker. I assume it now fills in adequately for the previously homegrown tech at layerci [0]?
Often times there are long-living processes which rarely change but take a long time to warm up. The Bazel [1] agent for C++ projects, the buildkit [2] state for docker, or the running Postgres or Redis server for a cloud native app for example.
It's why running "docker build" twice on your laptop is so fast, but running "docker build" in CI seems glacially slow.
> why is docker-in-docker a requirement, and how is that easier than qemu in qemu or qemu in docker or whatever?
The example given was running "docker-compose build", so you'd need either docker-in-firecracker (this post), docker-in-docker, or docker-in-qemu. You'd almost never run docker-compose build on bare metal in practice, because you'd immediately need to send the images you built somewhere else in order to use them.
[1] https://bazel.build/ [2] https://docs.docker.com/develop/develop-images/build_enhance...
I'm not experienced enough with Bazel to comment on that.
In terms of doing this in a CI env like actions where you may have different types of machines serving you, it may be problematic as the machine specs need to pretty closely match.
Docker in practice usually doesn't, but in theory it can and should.
A checkpoint/restore demo, of live-migrating running docker containers between servers, including running process state, was shown at Dockercon 2015. I can't find a video right now, but it was demoed then.
The "docker checkpoint" command has existed since docker 1.13.0 (released 2017), which allows checkpointing and restoring the running state of containers, including running processes, with some limitations. https://docs.docker.com/engine/reference/commandline/checkpo...
The reality is just unfortunately that these features are experimental, have been for the last 7+ years, and have taken far longer to mature than would be expected.
I’ve do all of the above using multi-layered Docker files and a cron CI job to rebuild the base integration test image every 6 hours. Sure if you need the isolation, Firecracker is the way to go. But if you invest primarily in container shenanigans to speed up CI with Docker, it’s not too much extra work to wrap it in a Firecracker VM, plain QEMU, or whatever once you start wanting more isolation.
Also, maybe I’m holding it wrong but Docker in Docker had not bitten us yet on our GitHub action runners.
We find the dominating factor in (our) incremental builds / CI to be network/io caching, which has less to do with firecracker/docker and more with the surrounding hw/sw (gha topology & smarts, IO speed, ...). It's a real problem in GPU/AI CI where we get monster image sizes. There were some cool blog posts ~last year on caching and routing tricks happening at GH (joint with MSR?), but they've seemingly gone silent..
I do love the idea of taking a snapshot of a prebuilt database image and can see where this would really speed up the tests.