Hacker News Reader

Comment by darren0 | Hacker News Reader

I completely agree. The current trend of orchestration all sells a story of hyperscale like GIFEE. That is not how most people do or should run there deployments (I'm talking about the vast majority outside SV). The operational complexity today is too high. Kubernetes is a big business system, it caters to enterprise. If you aren't an enterprise I would caution you against k8s.

View on HN

>> Kubernetes is a big business system, it caters to enterprise. If you aren't an enterprise I would caution you against k8s.

That seems overstated, frankly. What is so complex about, for example, GKE that renders it unfit for smaller organizations? If you mean "caution you against hosting your own clusters" I could probably get onboard with that.

For sure running k8s is difficult, I don't think anybody would disagree with me there, but of course you can buy GKE. I have to point out that GKE is the only decent k8s implementation, EKS and AKS are not real contenders. Digital Ocean will launch their service, I've seen it and it's good and I hope they do well. But in general you can't really expect the Kubernetes landscape to change much from what it is today, which is basically GKE good, everything else meh. Why? Kubernetes is not really a proven business model for cloud providers yet. It costs the cloud provider more to offer Kubernetes and it not significantly drawing more users in (and consuming more). The obvious example of this is AWS lack luster support and commitment to EKS. If there was a big business opportunity Amazon would be doing more. I have no concrete proof but ECS and Fargate stand to make much more money for Amazon (just a guess). So in general the argument that running on GKE isn't bad I don't think holds up because GKE is the only decent option. DO has yet to launch, but they can't really standup to GKE because you have to look at things holistically. Just running the cluster is like 10% of the whole solution. You need to then figure out how to do authentication, authorization, load balancing (built in service LB and ingress are not sufficient), monitoring, logging, and disaster recovery. Google has a product in each of these areas, but they have 100x more resources than DO. Basically nobody can (or is willing to) put the money into K8s like Google does and Google has yet to prove that their investment is worth it.

That's not to say I don't think Kubernetes will survive. I do think it will survive and it will be around for awhile. Because it's a good fit for enterprise right now. Not a good cloud provider business, not good for small dev shops, but good for enterprise.

It's interesting you didn't mention Docker once in all of that. I agree with most of what you said, but paying for Docker Enterprise gets you integration with k8s that makes the complexity tolerable and manageable, and also includes all that extra shit that needs to be included. Docker Inc seems to be all-in on this as a core part of their business, so for all the half-hearted ennui of AWS, Docker Inc seems to be heavily invested in solving the problem.

You make some interesting points, and I don't disagree with them (nor do I really have the inside view needed to even have an opinion on a good chunk of it). After reading your reply I have two thoughts: the first is that whether GKE is a profitable thing for Google Cloud at the moment, it definitely seems to have brought the platform a lot of positive attention. Secondly, I think you're right about the complexity of running kubernetes correctly. From that perspective maybe it's a useful analogy to think of it as an operating system for the data center, and that what is needed eventually are higher level abstractions that build on it and make it easier to deploy.

You know how these days, "Java Enterprise Edition on Glassfish" is criticised for being burdened with "cruft" ?

And this makes it large, slow to start, difficult to work with, and provides a bunch of obsolete features you might mistakenly start using, that turn out not to be as well designed as everyone thought, but that can't be fixed for reasons of backwards compatibility?

In 10 years, could people will look at Kubernetes the same way.

Rancher (2.0) certainly made it a lot easier for me as a sole developer to get Kubernetes running in an actually useful way.

Though I’m still running into a hundred gotchas a month :P

We're still on Rancher 1.6 and Cattle because it's much simpler for us and we don't have the time to invest in a migration to 2.0

I would suggest you wait until 2.0 is a bit more mature. It’s fun, but there’s still a lot missing (that appears to be in 1.6)

What do you think most people should use instead of k8s? Or is there not a good answer for that yet? Do you think containers themselves are overkill, or just k8s?

Containers have a lot of value. My rule of thumb is basically use containers (docker) and as little as possible of anything else in the container ecosystem. So basically put your app in a container try to not change too much from however you currently do things. There is no shame in running --net=host or just doing "-v /mnt/someebsvolume:/data." I personally like Hashicorp because their tools are simple and grow with you. Besides Hashicorp, I don't have great recommendations for tools. In my opinion the current container ecosystem is a disaster for users. Big business ideas took over and killed innovation. Simple lightweight modular systems all got annihilated by Kubernetes and now it's pretty much a monoculture.

Where can i read more about best practice regarding this while not getting to distracted or off in the wrong direction?

I would also really love to learn the same.

> Big business ideas took over and killed innovation. Simple lightweight modular systems all got annihilated by Kubernetes and now it's pretty much a monoculture.

I don't work with K8S or even docker on a day-to-day basis, but to me the ecosystem looks rather good. There are several solutions of orchestration and containerization competing from different vendors, and things tend to be standardised to make them work together :

- https://github.com/kubernetes/community/blob/master/contribu...

- https://www.opencontainers.org/

No later than yesterday there was that post on HN : https://news.ycombinator.com/item?id=18050781

People experiment, dissect, keep some old things, hack new things. K8S is still in active development, new initiatives like Istio and Knative are built on top of it. It looks like an healthy innovative ecosystem to me. I guess it depends of point of view and interests.

If you just need to "orchestrate" on a single box, Docker Compose (https://docs.docker.com/compose/) is the bees knees.

I still don't quite understand why Docker Swarm is so overlooked. It essentially offers easy-to-manage orchestration while providing most of the core features that many people are looking for. It even makes use of compose files, which many developers are already writing. Given that most of the complaints about Kubernetes relate to its complexity and the need for a Kubernetes "expert" to look after said complexity, I'm somewhat baffled by how many people just totally overlook Swarm as if it wasn't an option.

I think you are right but the fact that Docker Inc included Kubernetes surely didn't help Swarm adoption.

> I still don't quite understand why Docker Swarm is so overlooked.

No IPv6 support in it at all, and the company seems to have moved resources internally away from Swarm development.

Doesn't bode well for it's future. Expecting it'll be taken out to the back shed (-bang-) in a while.

Check this GitHub issue on SwarmKit for more info: https://github.com/docker/swarmkit/issues/2665

Docker staff have been saying words to the effect of "we'll make an announcement with info soon!" for at least the last three+ months, across several projects. Swarm, Docker-Machine, & Infrakit come to mind.

With no actual announcement(s) then occurring.

The commit activity graph on InfraKit - which people are supposed to use now that Docker-Machine has been deprecated - seems pretty clear:

https://github.com/docker/infrakit/graphs/contributors

SwarmKit is the same:

https://github.com/docker/swarmkit/graphs/contributors

Clearly these aren't where resources have been placed.

Actions are not lining up with the words given. :(

I guess you are right. It's unfortunate because judging from the comments here you would think that there's a market for a simpler orchestration software while it will be hard to compete as just another Kubernetes distributor (even more so now that there are several alternatives for the container runtime). But hey, what do I know about product management?! :)

Nomad is that. We run Nomad quite successfully at SeatGeek, and I know of several Very Large Companies™ doing so at scale as well. One of them even has a fruit logo.

You are using the enterprise version, I assume?

We are running the OSS version of Consul, Nomad, and Vault. While the enterprise versions provide some items that could be interesting, they are not necessary given our workflow, and I'd bet that the majority of development shops out there wouldn't need them either (though I am certainly more than happy to play with them!).

I have had a good experience using the Hashicorp stack of Nomad and Consul, integrated with nginx for load balancing.

This stack doesn't offer all of the same functionality as the Kubernetes ecosystem, but it has the benefit of being vastly simpler and composed of relatively small, independent parts.

We did examine Kubernetes, but for various reasons we need to manage our own cluster and the engineering overhead was too high for our team to take on.

Why is Nomad & Consul simpler that Kubernetes?

What about Rancher 1.6 and Cattle? Is it going to be improved? It had a perfect balance of complexity and features. I prefer it over k8s.

Reply on news.ycombinator.com