Hacker News Reader

Comment by amelius | Hacker News Reader

Anyone else getting totally bored from installing an reinstalling software environments over and over again into containers?

View on HN

Ops here, bored is good. Bored means it's working properly and it's a solved problem. Also, I haven't seen a good alternative outside ConfigMgr like Puppet/Chef/Ansible or something like Nix which has another set of problems. Containers work well enough and their positives vastly outweigh negatives in most cases.

It's also a solvable problem for most part. If you have a set of packages almost all the software needs, someone should probably create base image for everyone with everything pre-installed so your dockerfile starts with FROM companybaseimage:latest

> Ops here, bored is good. Bored means it's working properly and it's a solved problem.

It's not a solved problem because build systems keep breaking in random places.

Granted, these problems are in general easy to solve. But we're talking about death from a thousand papercuts here. In other words, boredom from a thousand compile errors.

:shrug: Thousand compile errors sounds like Dev Team hasn't buckled down and fixed their build script (Dockerfile). Maybe they should take some time and do that. Other beauty about containers is ability to build in environment Devs control and any changes to build servers shouldn't impact them.

Perhaps you have never been in the situation where one package forces you into a particular version which in turn forces its dependencies to versions, etc., eventually pushing you into an impossible situation.

Anyway, I wish I had a Dev Team like you have to solve these problems for me ...

If you aren't leveraging the namespace abstraction of containers, why use them at all?

Noting that I am being frank and honest here, but you don't have a dev problem here, but most likely an organization level problem.

Namespaces and interface contracts should be dramatically reducing dependency issues, not causing them to explode.

Every time I am dealing with containers, I wonder myself about the irony of Linus discussion with Tanenbaum.

No, not really but that is because I use containers as a way to create multi-purpose machines, not single-purpose application hosts. No docker or any of its clones except for those few services which are wholly based around it - e.g. Kasm [1] - but (bottom to top) Debian - Proxmox - Debian containers running multiple services. This lets me parcel out that biggish server into smaller bits which each play their own role: a database server running anything database-related (Postgres, Mysql, Redis, Rabbitmq, etc), a mail server running mail-related stuff, a 'serve' server running all user-facing services (dozens of them on a single container), an authentication server running LDAP/Kerberos and a central Letsencrypt certbot instance which serves all other services in my domains, a few build servers for different environments and, yes, a separate 'docker' server where it truly is 'containers all the way down, sonny'. Using containers this way makes for an easily manageable system where components can be swapped or migrated without significant effort without overcomplicating things just for the sake of it.

[1] https://kasmweb.com/docs/latest/index.html

> "a 'serve' server running all user-facing services (dozens of them on a single container)"

I'm no expert, but I've encountered the "stick to one process per container" rule of thumb many many times. Could you please share your perspective? Thanks in advance.

Because it's excellent rule of thumb. When you need to run more than one process in container, you are forced to write scripts/service handler. This introduces complexity to system and troubleshooting is much more difficult when things go wrong. However, on practical level, it has to be done occasionally especially when containerizing third party applications that were not designed for containers.

Ansible is directly your friend here. Puppet/Chef, and others help in other ways too.

Unless your problem/complaint is deeper then that.

Bored sysadmin: All I do is install X, Y, & Z over and over again. Surely there's more to life then this.

Resource-minded DevOps: Hey, it seems like we install the same 4 packages on these hundreds of containers! Surely there's something we can do to optimize this!

Even with scripting it is tedious because the scripts keep breaking with new versions of packages.

And the number of required packages gets bigger and bigger.

very true.

But with Nix!...Now you have two problems.

I haven't really ran into this issue that much, we've had the same setup across Ubuntu 18.04, 20.04, 22.04 and the only thing that has broken across that was setup of wireless.

If i have to do it more than twice, its being automated.

It's probably because everybody have been doing that for a decade and we have no good straight explanation on why.

The most naive thought it was good for security reasons.

It kept a lot of people busy during the ZIRP era when money was abundant or corporation were moving to the cloud.

My guess is either the industry is gonna stagnate or a new simpler solution to package and run application is gonna appear sooner or later.

Maybe try nix?

Does it have precompiled binaries for embedded systems?

My problem is that embedded systems usually have little compute power so compiling stuff for them is really tiresome (and cross-compiling is another nightmare), so I apt-get what I can and compile as little as possible.

Also, afaiu, Nix is tied to a particular version of libc which has a high probability of not working with the vendor-installed libraries on my systems.

Depends on the embedded system. It has precompiled binaries for aarch64-linux, but idk about others. There's definitely no public binary cache for bespoke, proprietary platforms though.

> Also, afaiu, Nix is tied to a particular version of libc which has a high probability of not working with the vendor-installed libraries on my systems.

Nix ships a whole dependency tree with every package, down to and including a libc. If you have another libc, Nix won't care.

On the other hand, if your hardware isn't supported by the libc Nix ships, the natural path is probably to package whatever given libc in Nix and build against that. Then you are back to building from source via Nix.

Nix has pretty good support for cross-compilation, multilib, and 'remote builders', though. You can set your embedded systems up so that Nix builds happen on more powerful machines and then get copied over.

Nix evaluation itself requires a lot of RAM, though, so if you use Nix for embedded you probably still want to push packages to the weaker systems from the outside.

I've been enjoying gokrazy. Makes cross compiling and updating my rpi server a breeze. Still an early project though but it works well for my needs.

So for you maybe not nix? I'd have never known given the original comment

And now you have two problems...

Isn't it always a matter of choosing which problems you prefer?

The problems that Nix gives you are much nicer to have than the problems of tbe FHS absurdity.

Reply on news.ycombinator.com