The build process would just create VM images with the required binaries in there and then deploy that to an autoscaling group.
It worked well and if you only ever intend to run a single service per machine then is the right solution.
Even without orchestration, I argue containers are useful. They abstract the operating system from the application and allow you to manage each independently. Much more easily than you'd be able to otherwise, anyway.
Plus you can run that image locally using something like docker-compose for an improved developer experience.
That does sound bit weird setup to me; with something like Packer building an AMI is almost as easy as building a Docker image with Dockerfiles so the benefits of using Docker seem quite slim?
That takes a TON of variables out of the equation.
I’ve been trying to pick the right Linux distro to base my images on. Ubuntu Server is the low effort route but a bit big to redeploy constantly.
I’ve also been looking at the possibility of using Alpine Linux which feels like a better fit but a bit more tweaking needed for compatibility across cloud providers.
Unikernels are also interesting but I think that might be going too far in the other direction for my use case.
For others doing this, I’d be interested in what distros you’re using?
Debian (and, formerly, CentOS) is a good standard: it occupies a sweet spot between ubuntu server and alpine, in the sense that it's batteries-included and very well-supported (apt/yum), but not particularly bloated.
I use debian for all my personal stuff. The BSD flavors (https://en.wikipedia.org/wiki/Comparison_of_BSD_operating_sy...) are tempting, though. Perhaps one weekend!
For other distros, if you're working on non-x86, or have uncommon dependencies, you should be prepared to pull in and build your dependencies from scratch, which is not much fun to maintain.
I tend to gravitate towards projects like BSD, since they align with my principles, but you do pay a cost in terms of compatibility with common software.
For example, Firefox does not treat BSD as a first class build target, so it's up to community members to build and deploy Firefox binaries, and report feedback on bugs that break BSD installations.
If access to the most up-to-date compiled versions of popular software packages is a big sticking point, BSD-land may not be the right choice. But if you're living mostly in `vim`, `bash` and `man`, and are willing to roll up the sleeves, BSD feels cozy.
It's maybe a 20% correct analogy (don't read too deeply into it, or you'll draw incorrect conclusions), but C++ is to GNU/Linux, as D/Rust/Zig is to BSD.
[1]: https://unixsheikh.com/articles/technical-reasons-to-choose-...
Thanks for the link - I’m taking a look at that now!
Note that my personal infrastructure is a mixture of Debian and FreeBSD and I dearly love both - if you forced me to pick one of the two to keep I'd be horribly torn.
In paper form (more gory details): https://www.usenix.org/system/files/conference/lisa13/lisa13...
They are using CentOS Stream now.
Our Jenkins creates a debian package, and deploying involves installing this package on a machine and creating an image. This image is then deployed to the ASGs. We operate in 5 regions in GCP.