Everyone is praising how well written this is, but I personally am confused: what do you actually GAIN by running a docker container without docker? Is it faster, or more secure? Why?
https://fly.io/blog/sandboxing-and-workload-isolation/
You can skip to the grafs immediately before and after the string "If you’re running someone else’s applications, you should probably care a lot".
I don't think you can reasonably host general-purpose applications on a multi-tenant basis on shared hardware using container systems (ie: using directly shared kernels), for reasons that post gets into. It's for the same reason that AWS wrote Firecracker to run Fargate, which is also a container hosting service.
It's my post and I don't totally get the writing thing either. My M.O. with these posts: write it like it was an HN comment, and then edit the sentences to be shorter. I'm glad people like it, though.
Well done.
Docker makes external dependencies management better, but it's an abstraction that leaks like a sieve when it comes to networking. If your /etc/docker/daemon.json file isn't set up just so, or your iptables are a bit off, or your vpn client is a bit too aggressive (e.g., Cisco VPN), the whole thing blows up.
Personally, I would rather manage dependencies than iptables rules.
> I don't think you can reasonably host general-purpose applications on a multi-tenant basis on shared hardware using container systems
I'm really not sure what point you're trying to make
If you’re running dockerd on one big server, you’re a single bug in either the kernel or dockerd away from someone in account A being able to attack account B.
This is also true about Firecracker or another VM approach but look at the relative exposed attack surface: if we’re on the same kernel, there are tons of different modules which I can attack — maybe even obscure ones you don’t or barely use - and if any of them allow me to run code, alter or leak memory, etc. mayhem ensues.
Running a VM reduces that risk because you’re sharing much less code & data but they traditionally added more overhead and startup delay, and if you’re deploying a traditional OS there’s a lot of management overhead. You’d see this as a security precaution but many people weren’t willing to pay for it.
Firecracker changes that calculation because creating a VM is fast enough and Docker means that you don’t need to support a full Linux distribution, just enough to launch the container image.
When most people don't understand the implications, offering a cheaper option that's almost guaranteed to inconvenience them (or be a catastrophe, depending on what they're doing) is not a fault that should be attributed directly (and certainly not solely) to them.
"don't think you can reasonably host" => I encourage our competitors to do this
"general-purpose applications on a multi-tenant basis" => any program a black hat who signed up with our automated system wants to run, next to all the other programs random people want to run
"on shared hardware" => one lump of iron owned by Fly
"using container systems" => containers vs VMs, in this case, and especially a VM manager that pays some attention to security.
What you gain is a stronger security boundary. Just FYI, since 2019, you can also do this in Kubernetes using Kata containers + containerd which will happily shim firecracker. The setup is not simple though.
https://github.com/kata-containers/documentation/wiki/Initia...
Overall, fly.io building infrastructure on this pattern and making it accessible is fantastic. Looking forward to seeing how this continues to evolve and am happy to see more infra build on top of firecracker. Very exciting!
I'm sure there is some advantage to all that cleverness but its not at all clear what that is (cost?). So, I disagree it the article is that well written. There's a lot of what and how but not a whole lot of why that matters to their customers. It kind of fails to communicate that part. It could be as simple as "you pay less for more because we did X and this is how we did it and why that matters to you".
Writing is hard.
That’s not entirely correct. You don’t run a docker container. The filesystem is extracted from an docker image and mounted in a VM (firecracker).
Big difference that requires the cleverness described in the blog post.
But in general case, their target should be the security.