Hacker News Reader

Comment by salamander014 | Hacker News Reader

I think people are missing the forest for the trees with this.

In my view, the reason Docker has all the hype is because I can look at a Dockerfile, and know what's up. In seconds. Sometimes in milliseconds.

It's a user experience thing. Yes, Nix is better for 'technical people that spent the time learning the tool', but Dockerfiles rely almost entirely on existing System knowledge.

Yes, Nix is 'better', but the fact is Docker is 'good enough' and also 'stupid simple' to get started.

Also Docker-Compose, I don't know why people hate on YAML. But it takes that same KISS attitude to build complex systems that can also be used as sanity checks for migrating to things like kubernetes.

Being able to spin up a complex full stack app with one command and a compose file that doesn't take any brain cells to read is worth it's weight in gold.

This is like the 'general case tool' vs 'DSL' debate. If it's easy to use, people will use it.

View on HN

Author here. As with most things, its all about the trade-offs. Docker has certainly proved itself and that approach has worked on a massive scale. However, its not a silver bullet. For us at Replit, our Docker approach was causing issues: our base image was large and unmaintainable and we had almost no way of knowing what changed between subsequent builds of the base image.

We've been able to utilize Nix to address both of those issues, and others who may be in a similar scenario might also find Nix to be valuable.

Of course Nix comes with its own set of opinions and complexities but it has been a worthwhile trade-off for us.

Those sound like issues with your Docker usage - there are options to keep base image quite streamlined (e.g. alpine or distroless images).

For context, I'm referencing our (legacy) base image for projects on Replit: Polygott (https://github.com/replit/polygott/).

The image contains dependencies needed for 50+ languages. This means repls by default are packed with lots of commonly used tools. However, the image is massive, takes a long time to build, and is difficult to deploy.

Unfortunately, slimming the image down is not really an option: people rely on all the tools we provide out of the box.

> For context, I'm referencing our (legacy) base image for projects on Replit: Polygott (https://github.com/replit/polygott/).

May I ask why you didn't use something like Ansible to build such a complex image? With appropriate package version pinning (which it's the real crux here) it should work well enough to get a reproducible build.

I understand it would already have been something different from a pure Dockerfile so it's not that fair to compare buuut...

> May I ask why you didn't use something like Ansible

They did; it's called Nix, and they wrote a blog post about it ;)

Correct, that's one of the cases where docker's layered image system doesn't work well. Nix is almost the perfect tool to perform incremental builds and deployments for the Replit requirements.

I wish that docker has the ability to merge multiple parent layers like git, then you can build the gigantic image by just updating single layer.

The only hack the docker can do is multistage-build, however that won't work reliably in some cases such as resolving conflicts.

Disclaimer: the following is still experimental, and will probably remain so for a while.

There is actually the --squash command that you can use during builds, to compress all of the layers: https://docs.docker.com/engine/reference/commandline/build/#...

For example:

  $ docker build --squash -t my-image .

In practice it can lead to smaller images, though in my experience, as long as you leverage the existing systems in place efficiently, you end up shuffling around less data.

E.g.:

  - layer N: whatever the base image needs
  - layer N+1: whatever system packages your container needs
  - layer N+2: whatever dependencies your application needs
  - layer N+3: your application, after it has been built

That way, i recently got a 300 MB Java app delivery down to about a few dozen MB actually being transferred, since nothing in the dependencies or the base image needed to be changed since, it just sent the latest application version, which was stored in the last layer.

Also, the above order also helps immensely with Docker build caching. No changes in your pom.xml or whatever file you use for keeping track of dependencies? The cached layers on your CI server can be used, no need to install everything again. No additional packages need to be installed? Cache. That way, you can just rebuild the application and push the new layer to your registry of choice, keeping all of the others present.

Using that sort of instruction ordering makes for faster builds, less network traffic and ergo, faster redeploys.

I even scheduled weekly base image builds and daily builds to have the dependencies ready (though that can largely be done away with by using something like Nexus as a proxy/mirror/cache for the actual dependencies too). It's pretty good.

Edit: actually, i think that i'm reading the parent comment wrong, maybe they just want to update a layer in the middle? I'm not sure. That would be nice too, to be honest, though.

> In my view, the reason Docker has all the hype is because I can look at a Dockerfile, and know what's up. In seconds. Sometimes in milliseconds.

Most of the times this just gives me more questions than answers, like: what does the entrypoint.sh file in this repo do? Only to discover a plethora of shell script for setting up the runtime environment based on different environment variables. Most of the time not aligned with any common standard or with how you generally would setup the application itself.

That’s because Docker just pushes dependency management to one layer below, doesn’t solve it.

But 99% of the times that's exactly what you need.

Well I think both Nix and Docker delegate the actual resolution of dependencies and it's not about implicit vs explicit dependency management, alone, it's more that with explicit dependency management you get reproducability.

And with reproducability you move the work from fixing broken builds, to implementing builds.

Of course, I can tag docker images and upload them to an internal registry, but that seems more complex to me, than doing this at the source level with Nix.

I think you are confusing a property of your expertise with a property of the tool. As someone who doesn't use docker all the time, I find it kind of a pain in the ass to read realistic dockerfiles or work with docker-compose. As a juxtaposition I found freebsd jails much more pleasant and sane to work with for security containerization. For deployment management I'm not sure if there are competitors to docker but it's not hard to imagine something vastly more pleasant to use.

Agreed with your opinion about `Dockerfile`. The article had me for a second until I saw the script code. I mean, my time is not infinite and I rather spend it to do things that are really important to me, not learning to write "yet-another build script" for a small system. So unless it's mainstream already, I'm not going to touch it.

`Dockerfile` is light enough for me to not hate it too much.

For the `docker-compose.yaml` story however, I can offer one reason: when you have so many variants(versions), so many setting options and so many data types (array, object, string etc), it's hard to find references to write one from scratch (have to read multiple documents to get it right). Your knowledge on docker command-line parameters does not translate to `docker-compose.yaml` smoothly(some option changed names, some don't work). And sometimes, some function works differently under docker-compose.

> I mean, my time is not infinite and I rather spend it to do things that are really important to me, not learning to write "yet-another build script" for a small system.

You don't have to jump into the deep end with Nix. If you're happy to just run shell commands (like Dockerfiles provide), then all you need is this:

    (import <nixpkgs> {}).runCommand "my-package" {} ''
      PUT YOUR BASH CODE HERE
    ''

No, please don't interpret it like this.

No matter what the format Nix script look like, it's still a script language designed to address something that has already been addressed (or can be addressed with light expansions). The very idea of "Hey let's build this whole new thing that does this specific old task a little bit better at the cost of learning many new concepts (and making many mistakes)" is not good at the core.

I would rather say, if the dudes there really wants to create a new language, fine, but at least make it big. By that, I mean don't just try to build a tool that preforms the old task a little bit better (at cost of learning), make a tool that does new things (in other words, "enables new possibilities") far better. Perhaps after that, the toolset could become something worth learning for.

(Currently, there are many ways to create reproducible builds. And even if you have reproducible builds, it does not mean the build will reproduce the same runtime result all the time. All factors combine, the benefit you can receive from the toolset is just not great enough at the moment. Hope you understand my point)

> The very idea of "Hey let's build this whole new thing that does this specific old task a little bit better at the cost of learning many new concepts (and making many mistakes)" is not good at the core.

You don't seem to have a problem with Dockerfiles, yet Nix was around for a decade before Docker existed. If you don't want people to reinvent things that already work, then your complaint should be directed at Dockerfiles. In fact, you should go and complain at the following projects, which were (a) created after Nix, (b) try to solve some subset of things that Nix can already handle and (c) are strictly worse than Nix (e.g. less secure, not reproducible, not cross-platform, tied to one language, etc.):

- Docker

- NPM

- Puppet

- Ansible

- Salt

- Webpack

- Grunt

- Gulp

- Homebrew

- Pip

- Conda

- Poetry

- Gradle

- Vagrant

- etc.

I'm a big fan of Docker-Compose so far because of it's powerful simplicity and it's introducing me to GitOps, Infrastructure-As-Code, and Terraform, all of which I'm really starting to like... and I really hate doing DevOps work.

I think your point is very valid, it has got to be simple and increase productivity instead of impede it. Using something better but getting stuck in the minutia every day is a waste, and not something anybody in senior leadership should ever approve.

Lots of people seem to be building containers with non-Dockerfile based things though, especially in the JVM world.

You mean through maven configuration? At the end of the day it is still a dockerfile but constructed using Maven's xml.

I hate it haha

We use Google Jib with Gradle (https://github.com/GoogleContainerTools/jib) and love it. It does some slight optimisations (just use the classes rather than jars) and removes some decision making about where files are laid out.

It also builds into the Gradle lifecycle neatly. I don't need a separate tool for building images.

I'm sure writing Maven xml wouldn't be fun though!

I thought at least some of those worked out without generating intermediate Dockerfiles or invoking "docker build". After all container images are just basically tar files with some metadata.

Or do you mean it's conceptually the same, just implemented differently? I agree there.

Reply on news.ycombinator.com