Hacker News Reader

Comment by justincormack | Hacker News Reader

Previously Compose did not have a specification, only the implementation, and it was tied to Docker's products. The aim here is to open it up, and let the specification evolve in new directions, such as better supporting deployment on Kubernetes, as an example. It is supported with some other implementations but we want to bring these closer together.

Compose has a lot of users, and hundreds of thousands of Compose files on GitHub, so we wanted to open up development as much as possible.

(I work at Docker)

View on HN

> better supporting deployment

Is there any "supported deployment" for docker? I had the impression that swarm has been sidelined, and docker-compose was discouraged. Is there a path to deployment on kubernetes that uses only compose? (I have a hard time seeing how that'll work with ingress, and "compose" well with other kubernetes deployments?)

We have some on-prem stuff that would work on single-node swarm - but I've felt that compose has been in a bit of limbo this past year or so... Is some kind of production style deployment comming back?

"Compose on Kubernetes"[1] seems to be what they're advocating at the moment, and it's included in Docker Desktop. It's probably an OK first step for simpler deployments, though I suspect in order to do any troubleshooting, one would have to learn k8s anyway.

1. https://github.com/docker/compose-on-kubernetes

To debug an application running on linux one does not need to learn machine code or assembly. There are tools that make it all possible.

Higher level abstractions can work and need to start somewhere. It really depends what tools are developed.

Have you tried moving from on-premise docker-compose to managed kubernetes in the cloud?

If you've been diligent in striving for stateless containers that will help you - but pretty much everything thing else in your compose file is useless.

Docker-compose gets you the "pod"-level in k8s, but doesn't really help with ingress/services etc. So no one can reach your app, certainly not via a load balancer that terminate ssl for you.

This is no longer a game of "beating the nginx container until it gives up and does what you need it to".

That's assuming the existing tools (implementation). Having a spec means you can build more and new tools that do different things. I would expect docker-compose to improve or something else that implements the spec can come along and do what people need. Or, more than one thing can come along.

Oh, I didn't read your comment with a view to the future - I certainly hope a spec and further development of the ecosystem can help improve the deployment story - which would help keep the development story relevant.

It was announced a ~month ago that Mirantis (Aquirer of Swam and Docker Enterprise) will continue to develop Swarm with no implied death date. https://www.mirantis.com/blog/mirantis-will-continue-to-supp...

It is also an open source project, so even if the corporate overlords abandon it, the code could live on!

I certainly hope so - Swarm is a wonderful orchestration system for small deployments (at least, I haven't tried to use it for large ones), and is really easy to use. It's also vastly simpler to comprehend and operate that Kubernetes, and doesn't eat CPU & memory for breakfast. I'd wager that Swarm would be a better fit than Kubernetes for most systems.

Totally agree. I tested out both Kubernetes and Swarm for a recent project. Swarm was so vastly easier to not only understand at an architectural level, but to actually use, that I was delighted by using it. Kubernetes, on the other hand, was a real pain, and overly-complicated for 90% of projects I would say.

It was even very difficult to setup a test cluster on my MacBook Pro to try out multi-node support (you have to find a good distribution, something that supports Mac for multi-node, etc.). Whereas for Swarm, it was super easy.

> I certainly hope so - Swarm is a wonderful orchestration system for small deployments (at least, I haven't tried to use it for large ones), and is really easy to use.

I would argue that Docker Swarm is a wonderful orchestration system for all practical real-world applications, with the notable exception of really global and gigantic multi-region deployments.

Docker Swarm works superbly out of the box and it's trivial to setup a multi-node heterogeneous cluster of COTS hardware and/or VM instances.

I think there's a more specific statement about k8s vs swam. Swam is much better at running other people's applications that you're treating largely as a black box -- k8s excels at running your in-house app. The simpler deployment model means there's far less environment specific weirdness you need to contend to to make an arbitrary app run.

The systems I've deployed with Swarm have generally been a mix of both - not sure why it would make a difference if you were running in-house or off-the-shelf images?

That sounds promising. With the only choice for a while seemingly being "single container" or "k8s" for deployment - docker-compose for development seemed a bit of a dead end (ie - might as well go back to vagrant, and at least get proper heterogeneous host support as everyone would be running VMS, with no need for a Linux kernel on host).

With an option like swarm, that fits a certain on-prem type scenario (eg: what gitlab now does with its horrible (working, but still horrible) omnibus installer would be a good fit for swarm).

I never did understand why docker-compose wasn't treated "better" by docker - I guess they focus on non-linux devs? Docker without an orchestrator is a bit like a daemon without a supervisor - if everything fits in a single container - why not just run it as a Linux process?

And compose was a perfectly OK orchestrator - it did some minimal, sensible things that you need, to compose a couple of containers. But it felt scary that you never knew what upstream intentions were.

> docker-compose for development seemed a bit of a dead end (ie - might as well go back to vagrant

It might be a VM either way for Mac and Windows, but Windows developers are likely to always be running a VM with Docker integration soon due to WSL 2, and on Linux it's still an incredible productivity multiplier that beats the pants off everything else when you have dependency issues. Only MacOS is left behind with the worst developer experience out of the three.

I actually meant that with vagrant, even us on Linux would run VMs, so if I set up a dev env, I can be pretty confident a colleague will run the same thing, with similar characteristics, on windows or Mac. With Docker, I feel it's still a bit kinda-sorta. Especially if you need to run multiple stacks for switching between projects.

> With Docker, I feel it's still a bit kinda-sorta.

What makes you think that? Only difference I can think of is use of hemorrhaging edge kernel primitives in your application. Even networking is handled by docker-compose and you don't have to worry about port collision.

> With Docker, I feel it's still a bit kinda-sorta.

This was the first time I ever heard anything of the sort. Docker has been not only solid but by far the most performant system available, and running stuff on VMs always felt a half-baked solution in comparison.

Can you shed some light on what led you to form your opinion?

A former colleague of mine used to rail on Docker for MacOS a few years ago. He always complained about it being super slow to sync with the local file system. I never had any issues but was running everything on Linux. Since then I've moved companies and now I'm running docker on a Mac. Haven't noticed any eggregious performance issues, but as I understand under the hood it's still basically running in a VM.

Docker is not in the "supported deployment" business any more. We sold Docker Enterprise to Mirantis, to focus on developer tooling. The aim of open sourcing Compose spec is to help make it easier for people to make better deployment tooling, and to clean up the parts of the spec that were specific to the methods we had before to make it more general.

There are quite a few ways now, and we will be helping make them better. These include Kompose (the maintainer, Hang Yan, is involved with Compose spec now) which converts to Kube Yaml, some work on Compose-Helm integrations that is ongoing, Amazon ECS has Compose support, compose-on-kubernetes (we think this needs rewriting to use a CRD again, after having changed it to use API extensions), and some other projects which are being worked on that I have heard about.

What is Docker's business model these days?

Hi Justin, I work at Heroku for Salesforce. I'm unsure if you've worked with anyone here in the past (like on our older app.yml, heroku.yml and constellations, etc stuff). But I just wanted to say thanks for establishing a standard for this and helping the rest of us in this space continue to evolve.

I'm super excited for this. Back when Docker Compose was still called Fig, I got annoyed at how it handled volume-only containers for databases, and wrote Dup [1]. Nowadays Compose is way WAY more useful, but there's a number of projects that still rely on it -- the spec means I can write a converter for the config files (from Dup to Compose) and be sure that it will work :)

[1] https://github.com/girvo/dup

Does this spec carry forward the v2 stuff into v3 that would otherwise require --compatibility to function without swarm?

So far we have not made changes to the spec, just carried it forward but we definitely want to start cleaning up the v2 v3 mess, and make a single spec work for all platforms.

See https://github.com/compose-spec/compose-spec/issues/12 and also https://github.com/compose-spec/compose-spec/issues/13 for making it feature not version number driven.

I would be surprised if there is the political will at docker to do this when it wasn't a priority for compose itself

Docker (Inc) has new priorities and different influences. I'm inclined to feel some optimism that this shifting focus will have beneficial repercussions in places like this.

This. My current usage is a muddy mess of v2 and v3 specific features an some require the compatibility flag while others don't.

Hi, I would like to use docker-compose without using docker. Specifically, I’d like to run devil box (php development environment) on fedora using pod man and docker-compose. Is that possible now?

Reply on news.ycombinator.com