Hacker News Reader

Sometimes the proof is in the pudding.

I’ve had similarly adverse reactions to the seemingly “unnecessary” complexity of many things: docker, webpack, autotools, react, AWS, just to name a few.

But I’ve found that, upon closer inspection, the software is usually complicated because the problem domain is complicated. Further, as an inviolable rule, the most popular tools for any problem are the ones that don’t make you rewrite all your code; popular tools build on top of the mess we already have. This can give the illusion that the tool is somehow responsible for all the preexisting complexity. Tools which attempt a “clean slate” approach are mostly doomed for obscurity.

In the case of docker-compose, I can say after years of resistance, I tried docker compose and was an immediate convert. Setting up, documenting, and sharing development environments has never been simpler.

I would add a caveat: Docker compose is nice for setting up development environments as long as everyone's developing on the same platform.

I work in a shop where various developers work on Linux, OS X and Windows (useful for dogfooding - we support deploying to multiple platforms), and, there, Docker Compose has ironically turned out to be our chief source of cross-platform headaches, due to annoying differences in how it works on each host platform. Particularly with the networking subsystem.

Everywhere I've ever worked that had docker/vagrant, the devops guy has promised me easy, one command environments, and not once has it worked.

I've seen the product they are selling, I've watched a docker command build me a multi-part infrastructure, automatically source databases and assets from afar, and do it all quickly. But then it failed the second time I tried it. I've not found it to be a reliable tool at all.

> Docker Compose has ironically turned out to be our chief source of cross-platform headaches

Which is the whole point of it. If it's not, it's just a bad abstraction. Docker-compose itself becomes the only variable

Perhaps I should rephrase:

Since moving to Docker, we now have more cross-platform problems than we used to.

We continue to use it anyway, because there are some other benefits that we feel outweigh shortcomings like these. But still, it's annoying.

Even in a shop with just Linux and mac, I think docker (especially deployed via fargate) causes way more problems than it solve. We have a single instance of a single app that used to run just fine in a Ubuntu vm. Now it runs in fargate. We waste huge time futzing with the image size due to fargate insane 8gb limit. We wast huge time working around not being able to get a shell when something goes wrong.

Local development also sucks. The docker engine is essentially unusable on my colleagues Mac, consuming 100%+ of all available cpu while sitting idle. On my Linux box, either docker breaks my networking, or IT firewall rules break dockers networking. It’s even worse with wfh, because our vpn is incomparable with docker. Local dev just happens in an anaconda env instead. So what’s the point?

VMs have none of these problems. Docker is the leakiest abstraction I’ve yet to come across.

I'm not incredibly knowledgeable about AWS and Docker, but I have a couple questions for you:

- How would EC2s compare to Fargate ? I encountered situations where, running the numbers, I much preferred having my own set-up with images ready to deploy + adapting my EC2 instances to the task, instead of dealing with Fargate's convoluted restrictions and definitions

- Has your colleague tried to increase the disk size given to Docker? I have seen on a few macbooks that this made a world of difference for CPU usage, and I thought I'd pass on the information if it hasn't already be tried yet

To be fair it's an issue with docker, not compose (volumes, networks differ in implementation between platforms)

Everyone is so obsessed with never repeating themselves that they've forgotten to ask whether they are still removing duplicated effort instead of just duplicated text. Every tech cycle we get people trying to compress the text long after the power of the system has already been maximized.

Particularly, as a species, we discount the future heavily, so as developers we do not accurately estimate the variance introduced by something that works by magic until it doesn't. So on some unnamed day in the future when you had other plans, you will be completely blocked while having to learn a bunch of things we were avoiding learning in the first place.

> Am I just totally wrong here?

Yes. I think so.

docker-compose is just a tool to run multiple containers together. It is extremely simple, readable and straightforward (if you're familiar with docker). It is not an inf. abstraction at all.

It's not extremely simple. Do you know when to use v2 vs v3?

Looking into it for a minute I don't see any particular gotcha. v3 is a breaking change compatible with Docker Swarm, so it's just like any versionned tool: if you're already on v2 and don't care about the new feature stay there, if you're on a greenfield project or do care about new feature use v3. Am I missing something?

It's like asking when to use python2. Use v2 if you have no choice. If you're asking, apparently you have the choice, so there's your answer.

Nope, entirely wrong! v3 isn't an upgrade, it has features specific to orchestration. v3 is a fork, not an upgrade. v2 is better for local development, it has things like depends_on which v3 doesn't have.

depends_on was removed because it was an inconsistent and problematic directive. You can abuse links instead.

You should only use v3 if you need v3 features and know how to use them. You should use v2 for local development if you're not using stack/swarm locally: https://github.com/docker/compose/issues/4305#issuecomment-3...

Docker is how you put your widgets into a container, docker-compose is how you put a few containers on a ship. How is separating those responsibilities overengineering?

I think you generally have the right sentiment, but some of these solutions are born out of necessity. Using containers at scale is really hard when you're trying to solve for optimal resource usage and resilience. These abstractions may seem crazy but other solutions are probably much crazier.

The other part of this problem that isn't applicable generally to other things, is this is a fairly unique problem to cgroups. By design cgroups don't have a singular implementation or solution, so there are a lot of "competing" solutions. I think some amount of unix philosphy is at play here, to do one thing (or really, fewer) and that's why it seems insane how many systems you have to hook up together to make the thing work the way it should.

I'm fairly devops centric, but I'd still take this with a grain of salt b/c containers aren't my area of expertise.

Maybe you have a point but from my perspective you're overthinking it. Lots of developers use Docker Compose and will continue to do so because it's a great product. The news here is that there is now an open specification that others can contribute to.

You're not wrong. What you are missing is how much complicated web development (and software development in general has become). Even if you are doing a simple Static website it's more complicated than yesterday. If you are using Git to push your changes, you already need to push to Github and then SSH into your machine, make a pull request.

Oups. Now Nginx is still serving the old version through cache. So need to clear cache, restart, shut this, reconfigure this, etc... After a while this gets tiring.

Docker is not the perfect solution (one can wonder if it's actually a solution) but it's a symptom of the real issue rather than the issue itself.

It is just a matter of time. Right now, we’re in the thick of things and as pointed out by others, the problem domain is very complex. As we move forward, though, our solutions will become simpler and our container tools will work more elegantly.

it is but it solves the distribution problem. golang aimed to solve the same problem by compiling to a single statically linked binary and it was one of its biggest selling points. i feel docker blunted this part considerably, but it also benefited it nevertheless: development environments, shipping runable packages with assets, etc. is easier when you literally have to only do docker run whatever.

>I'm not a devops person at all

You should try it before you knock it. There are a lot of flavors but the abstractions of any single flavor isn't that deep.

Containers themselves aren't even an abstraction, they're just namespaced apps.