Comment by banana_maker | Hacker News Reader

Comment by banana_maker | Hacker News Reader

Parent Full thread

For people who use docker at work, avoiding it is simply not possible. Our stack revolves around images (not necessarily docker), so *bsds are dead in the water for me.

Additionally, orchestrating is simplified with docker-compose vs managing many jails. I used to manage freebsd jails via cli in FreeNAS, but orchestration with docker-compose is much easier and trackable in git. Transferring between machines is as easy as setting up docker, git cloning, and setting secrets. [0] Podman solves some issue docker has, but using stuff like S6 [1] in containers helps a ton. Perhaps most importantly, docker images are reproducible (for the most part) while jails only have templates, so it's up to you to manage reproducibility.

Don't get me wrong- OpenBSD and FreeBSD are amazing distros. OpenBSD has the best user experience in my opinion, which is why I use it for my router. But they suck for modern gaming and stuff like docker.

"custom-compiled nginx version downloaded off some guy's FTP"

This is a strawman argument. But, sometimes one might want custom compilation without installing a host of build tools on the host system. Or one might want to have a reproducible build not tied to the host system. Compilation may be expensive (like with ffmpeg) or the host may be underpowered like a Raspberry Pi. Etc.

[0]: https://github.com/andrewzah/lilac-docker/tree/main/services

[1]: https://skarnet.org/software/s6/

I agree with your arguments in that if your stack is already built on Docker it's a lot to ask to move to a different system. But if you are choosing a new stack for a new project, why not look at alternatives to Docker? There are many and jails is a good one. Granted, I haven't looked much at orchestration tools for jails.

As far as do you need Docker in the first place? Well maybe. One of my favorite orchestration and deployment systems I built was based on packaging everything as .deb files and running our own apt repo. Since all workstations ran Ubuntu and all servers ran Ubuntu getting our system up and running was as easy as adding our custom repo and running `apt-get install our-custom-project`. apt is great for resolving dependencies and this way we don't end up with a mess of files all over the place. Plus this way we got all the benefits of not having to update every container when a libssl update was required. Just run `apt-get update && apt-get (dist-)upgrade` and suddenly you are fully up to date and restarted.

Orchestration on this system was accomplished by using puppet to set up the custom repo, install the packages, install all the current config files for the system services as well as our own, and starting all the services in order. Reproducible to the point where when one of our servers (we had a few pieces of beefy physical hardware) blew up, we simply set up a new one, ran the puppet manifests and were back to full capacity within like an hour. Mind you this was back in 2010-2012 and tooling has only gotten better since.

This type of thing also allows you to nicely package any custom versions of software you want as well. Want a custom build of nginx? Go run the script that builds it and makes a .deb out of it, then upload to your repo. You aren't relying on some guy with a blog post to keep his server up. You aren't even affected by GitHub going down if you don't host your apt repo there. Or use it out of a PPA someone else maintains. But there is zero need to wget/make/make install with this setup. You aren't doing reproducible builds because it's a build once, run everywhere system. And it makes you very directly consider what your dependencies are. Do you really need that unmaintained library written in an esoteric language that requires a SPARC to compile? Docker allows you to hide bad dependencies behind the idea that they are inside a container so the harm they can cause is limited and the headache is localized. But that just treats symptoms, not the problem.

"Since all workstations ran Ubuntu and all servers ran Ubuntu"

Admittedly our company is small (~5 fulltime devs), but we have: mac osx catalina, debian buster, debian bullseye, and ubuntu bionic beaver. So precompiling .debs won't work here. Docker gives us all a common ground, minus some wonky mac docker issues with DNS.

Also, this assumes that one is using a server in the first place. We run our own kubernetes cluster that we automatically provision and deploy pods to, so there is no server to upload files to.

"You aren't relying on some guy with a blog post to keep his server up. <...> And it makes you very directly consider what your dependencies are. Do you really need that unmaintained library written in an esoteric language that requires a SPARC to compile? Docker allows you to hide bad dependencies behind the idea that they are inside a container so the harm they can cause is limited and the headache is localized."

Again, this is a strawman. You can butcher things with docker, or without docker. The same can happen with i.e. Ansible & Terraform (which we also use). I can, and do, analyze our images to see what we can reduce to. Most of our images are either on Scratch or Alpine Linux, thanks to multi-stage builds.

Since each build is localized to a container, we can independently update images and not have to worry about dependency mismatches, or random directories being modified, etc.

My opinion will be biased because I've written at least ~120 docker images in the last two months and spend a good bit of time tweaking and optimizing them.

> Admittedly our company is small (~5 fulltime devs), but we have: mac osx catalina, debian buster, debian bullseye, and ubuntu bionic beaver. So precompiling .debs won't work here. Docker gives us all a common ground, minus some wonky mac docker issues with DNS.

That does make it more difficult. Docker does sound like the common ground then.

> Again, this is a strawman. You can butcher things with docker, or without docker. The same can happen with i.e. Ansible & Terraform (which we also use). I can, and do, analyze our images to see what we can reduce to. Most of our images are either on Scratch or Alpine Linux, thanks to multi-stage builds.

It's a related argument. My point is that Docker allows you to take shortcuts too easily compared to other methods. And when you are faced with figuring out how to make your software work with widgetlib 1.0.4 provided by the system instead of widgetlib 1.0.5 which is what you originally built it for, you have a choice of packaging 1.0.5 yourself and potentially doing that improperly (make && make install inside a Docker container, paying no attention to dependencies or upgrades), or properly (by creating a standard reproducible build you can track). Docker allows you to take the shortcut easily. It's a powerful tool and it does allow you to create good images, but I have also seen some terrible ones (just like I've seen bad examples of .deb packages, but a lot fewer of them).

Regardless, it's about how you work and how you structure things. I am coming around to Docker as a workflow, but I doubt I'll be creating 120 microservices to run one project anytime soon. Too many things to keep track of and update.

Reply on news.ycombinator.com