Hacker News Reader

Comment by tyingq | Hacker News Reader

"Kubernetes is a component (like containerd or swarmkit)...Docker is a platform...(like Cloud Foundry or Openshift)"

I think that's pushing k8s farther to the left of what it really is, and pushing Docker farther to the right of what it really is.

k8s, for example, incorporates service discovery. As far as I can tell, swarmkit does not. k8s incorporates networking, containerd does not. Similar for things like ingress load balancing.

There are certainly potential customers debating, directly, k8s vs your full Docker platform, even if there are some gaps they have to fill with other software.

View on HN

Especially since OpenShift uses both Docker and Kubernetes...

So Docker is like Openshift, except openshift uses docker as a component? Very confusing..

Openshift uses a very small subset of Docker. That subset is now available standalone as containerd, in part to allow Openshift to no longer depend on what is essentially a competing platform. I expect lits of constructive open-source collaboration with Red Hat on containerd.

This is why it's important to clarify what "Docker" means in context. Kube and OpenShift DO use Docker (the lower-level infrastructure parts of it, but not the higher-level platformy parts of it).

That represents the evolution of the containerd-vs-docker split, and (to me) seems totally rational.

> k8s, for example, incorporates service discovery. As far as I can tell, swarmkit does not. k8s incorporates networking, containerd does not. Similar for things like ingress load balancing.

Swarmkit does in fact implement service discovery, networking, and ingress load-balancing. It also implements out-of-the-box node security and mutual TLS, secure secrets management, a built-in raft store, infrastructure-agnostic overlay networking, and various goodies which we needed to make Docker work great out of the box.

containerd is a different type of component entirely - in fact it is very complementary to kubernetes.

> There are certainly potential customers debating, directly, k8s vs your full Docker platform

They typically debate Docker vs kubernetes-based platforms (among other possible alternatives). If they're a Red Hat shop, they typically evaluate Openshift. Sometimes there's a team building an in-house platform. Nobody ever deploys kub alone in production. There is always some form of platform on top.

>containerd is a different type of component entirely - in fact it is very complementary to kubernetes

You made the initial comparison.

> Nobody ever deploys kub alone in production. There is always some form of platform on top.

Which is another way of saying "even if there are some gaps they have to fill with other software". Sure, some customers pick a platform where the gaps are prefilled. Not terribly different from some of your customers that pair docker pieces with pieces from other vendors.

>> containerd is a different type of component entirely - in fact it is very complementary to kubernetes

> You made the initial comparison.

Yes, sorry I wasn't clear. I meant that they were both components, as opposed to complete products.

On the other hand, SwarmKit and Kubernetes are comparable in functionality.

> Which is another way of saying "even if there are some gaps they have to fill with other software".

I agree.

> Sure, some customers pick a platform where the gaps are prefilled. Not terribly different from some of your customers that pair docker pieces with pieces from other vendors.

I also agree.

It seems that there is nothing left to disagree on :)

Sure. Said more concisely, my beef was that you compared k8s to things narrower in purpose, then compared Docker to things broader in purpose. It made the gap between the two seem larger than what it is. I agree there's a gap.

Edit:

Re: "Try developing on Docker for Mac/Windows and deploying to production with Docker for AWS/Azure"

Developing on a platform that isn't the same as your production platform is an avoidable situation. I could develop on .Net and deploy on mono as well, but...

The gap is in fact huge. Try developing on Docker for Mac/Windows and deploying to production with Docker for AWS/Azure. Then try to build the same level of functionality yourself from naked kubernetes and only open-source tools. Let me know when you're done :)

The difference is that I can put together components of Kubernetes that actually work. Sorry to be inflammatory, but I've been working on the bleeding edge of Swarm and it simply doesn't deliver what's promised (ex: flaky management of iptables, missing idioms/abstractions, etc.).

Single-host Docker in production, yeah that's not terribly complex, but the new price of admission for production is a complete orchestration layer, and there is 0% chance I consider taking a Swarm cluster to production any time soon. And that's to say nothing of the DDC disaster...

MiniKube as a local env is getting great feedback filling this niche.

It's absolutely awesome in this role.

I don't know if a few hundred lines of shell scripting and Helm charts are considered building around it though, but that's enough for a Git hosting service (Gogs) and a CI (Drone 0.5) with persistent storage with vboxsf.

I deploy on minikube (Mac Desktop) and GKE.

There are 4 lines of configuration that differ between those two environments.

> Nobody ever deploys kub alone in production

That doesn't gel with any reality I know of, sorry. Lots of people use higher level platforms, or develop bespoke tooling, to be sure, but that's going to be true forever, for every platform.

(disclosure for others: I'm a Kubernetes founder)

Replying to myself: The fact that someone builds tooling on top of your offering is not an indictment of your product. It simply means people are using it.

Be worried when people STOP trying to wrap your stuff with their own opinions.

> Nobody ever deploys kub alone in production.

Uhh, we do. And we're not alone as we work with several other companies that do. That's quite a big nobody tent.

What do you use for provisioning and upgrading Kub? For monitoring it? For CICD? For access control? Image registry? Image build? The collection of these things, and everything else required for deploying and managing containers in production, is what I call a container platform.

Is kub core not working on spinning out built-in volume plugins out of the core to keep it smaller? Are the maintainers not recommending using third-party tools + 3PR for all major new features going forward?

It seems like my comment was misinterpreted as a criticism of kub, or a sign of ignorance. It was neither. I stand by my comment that nobody runs naked kub in production. That doesn't make it any less good, stable or useful. It just means it's not meant to be a complete product.

I beg to differ, having done Kubernetes training for nearly a year now. None of the shops I've trained at had any platforms on "top" of Kubernetes. They all ran Kubernetes directly from the source. And we're talking about some big enterprises here, not mom and pop shops.

When an enterprise runs Kubernetes directly from the source how do they deal with bugs and support ? Doesn't that need some vendor backing with tight SLAs ?

> Nobody ever deploys kub alone in production.

We do: https://kubernetes-on-aws.readthedocs.io/en/latest/admin-gui...

There are only a few AWS integration components we added, e.g. to use the "new" AWS ALB (see the linked text for details).

> Nobody ever deploys kub alone in production.

We do. I work at SAP, and we run our company-wide OpenStack on pure Kubernetes on CoreOS's Container Linux. [1] We do use Docker (the container runtime) because it comes with CoreOS, but no other Docker product as far as I'm aware. I've been working with Kubernetes for quite some time now, and honestly don't know what else you would need on top of it (except for some Continuous Integration tool, of course, but that's already a staple of any well-organized agile team, no matter the platform).

[1] I mean the API and orchestrator parts, not the customer VMs themselves. These sit on traditional hypervisors.

Of interest, SAP is a Platinum member of the Cloud Foundry Foundation and has a certified CF distribution, SAP Cloud Platform.

Disclosure: I work for Pivotal, another Foundation member, which also sells a commercial CF distribution.

Ah, yeah, the CF guys are from another team. They deploy their stuff on (among other things) the VMs created by the OpenStacks running within our Kuberneteses, though. It's turtles all the way down. :)

That's more turtles than I've typically seen. How're you deploying k8s?

Bare-metal with homebrew automation. We install CoreOS via PXE boot, and during the installation it also sets up a Kubelet as an rkt container. The Kubelet then spins up the other k8s components via manifests. The pod and service networks are routed via BGP using our own https://github.com/sapcc/parrot

Later this year, we will go back and evaluate the maturing k8s administration landscape. Our current approach has a few drawbacks, e.g. it requires a CoreOS reinstall to upgrade k8s cleanly (since all the magic happens in cloud-init and Ignition).

Could you email me on my work address? I may have something of interest to share. I'm just not sure if we've made it public yet.

See my clarifying comment: https://news.ycombinator.com/item?id=13788426

Reply on news.ycombinator.com