Hacker News Reader

You are correct on Google self-interest in the matter. Not so sure that Google's self-interest is so divergent from the average developer interest.

Kubernetes is hard to run by oneself. Worse, self-cobbled setups are also hard to run by oneself. Distributed systems, that is systems that require uptime and/or more than one computer, are hard.

The fact of the matter is that computing is the new electricity. While one can run oneself's own generator in the backyard, it is more efficient to buy electricity from a big provider, small-scale solar notwithstanding.

Kubernetes offers a well designed abstraction that simplifies managing distributed systems at most scales. Bonus, it has become a shared standard among providers of computing. That is a big step beyond proprietary lock-in abstractions [AWS, etc.], and likely close to the best one can hope for, technically speaking. Economically, perhaps a future solution is regulating computing providers as utilities.

I agree that running kubernetes ourselves does actually serve our own interests. But I still think it's much more of a slippery slope than alternative solutions (being that some of them are not developed with the intention to eventually get you to use the hosted version)

What are the alternatives of which you speak?

Which part of kubernetes is "developed with the intention to eventually get you to use the hosted version"?

Offering software for free, software that is widely adopted cannot be based on the motivations you propose, unless there is a grand conspiracy of cloud providers for which I am unaware.

> Which part of kubernetes is "developed with the intention to eventually get you to use the hosted version"?

I believe it's all of it. Why else would they spend money on building and promoting it?

So it is evil unless I can prove otherwise? Your straw man has been exposed. https://en.m.wikipedia.org/wiki/Straw_man

You don't have to prove anything. Nobody is on trial here, I hope :)

I see a semblance to their original motivations in building chrome. By making the web as a whole more competitive, it made google's offerings more competitive (more ad surfaces, more gmail users, less OS lockin). Actually winning the browser war isn't necessary.

For a huge number or organizations, the cloud isn't a meaningful option because the switching costs are astronomical. Kubernetes, otoh, means someone that chooses on prem today actually has a migration path later.

Google benefits hugely from that path just existing.

LXC on which Docker was based always had a much more sensible container model and contrary to the pervasive misinformation by the devops ecosystem is far easier to use.

It supports a standard OS environment and does not enforce the use of layers so users don't have to deal with single process non standard OS environment which removes half the complexity.

We are working on a project, Flockport, that supports LXC and provides an app store, orchestration, networking, distributed storage, service discovery and HA. So there are attempts to explore simpler alternatives.

I have the same question as the other commentor - what are the alternitives? Docker compose? Apache Mesos?

Can you please explain what you think a better solution is? I am honestly curious to know.

I think Google's interests are aligned with developer interests here. The main benefit to GCP that k8s success brings is that migration between clouds becomes easier (e.g. moving from EKS to GKE would presumably be easier than moving from Elastic Beanstalk to App Engine). Less lock-in benefits everyone except the current market leader.

The stateless services (and cloud native stateful) become portable, the new lock-ins are their value added data-stores (spanner, aurora, cosmos).

Well I agree that the interests are aligned insofar as Google is trying do dethrone AWS which currently has a near-monopoly on cloud. But I'd still rather live in a world where the compute fabric of the internet is not centralized in the hand of a few corporations. I'd much rather get paid a salary from money that would have otherwise gone to a monthly *-Cloud payment. And I believe that in a lot of cases, the non-Cloud version of the product would actually be a better, cheaper, more reliable solution. But that will become a harder and harder sell the more successful the marketing from cloud providers is.

K8s only gives you theoretical leverage in negotiations with "cloud providers". As long as you can't reasonably run it on your own hardware (because it's simply way too complex and you'll be having trouble finding experts, even at obscene salaries) you won't be realistically able to migrate to another "cloud provider" either.

It's an old trick from the playbook of ERP software vendors: make your software absurdly configurable so that all the meat is in the configuration.

Worse (and I know this isn't popular and will hurt) k8s for all intents and purposes just runs Docker images which are just pointless wrappers that don't add anything but point-of-failures and vulnerabilities.

A container is just a linux process with special restrictions. Its not just a pointless wrapper.

Don't think the term "container" is really well-defined.

The "container" that docker and others implement is actually a collection of different kernel namespacing features. I assume the one you are referring to are cgroups. I think a better description would be that each process in a linux system is part of (many) cgroup hierarchies. And you can have more than one process in each of the groups.

I think what parent meant is that you can actually get all of these really nice isolation features for your service without using "Docker". It is trivial to enable them using linux command line utils, or use something like systemd which can also do it for you.

I meant all the namespacing features.

My main argument was against a container being something "more" than just a standard linux process in the sense that "more" is extra software that wouldn't otherwise be ran.

Of course Docker is just the popular brand name for those isolation features and 'image' 'container' and all related terminology exist independently without Docker

Docker is also a cross platform command line tool that helps to manage said images and automate a lot of work that would otherwise need to be done. For me it is kind of like Dropbox: yes you can piece it together yourself, but using it is very convenient and you can spend your time elsewhere.

In my opinion its on its way out. CNCF and k8s has already basically replaced it with the CRI initiative and containerd[0][1]. Runc, rkt, and a multitude of other tools run containers. img, ocra-build, and others can build them.

[0] I realize this is from Docker as well but I feel it supports my point that Docker, Inc themselves are shedding baggage to still stay relevant.

[1] https://kubernetes.io/blog/2018/05/24/kubernetes-containerd-...

I'll have to look into it thanks. I'd like to not end up requiring different tools to run the same container on Windows/Mac and Linux. Currently my workflow is easy, I create my Dockerfiles on a mac, do all of the building and testing, and then just tell devs on Windows to pull a repo and do a docker-compose --build up. I hope in the future this will not grow into tool hunting.

You'll probably be able to continue using the same tools across platforms to run containers.

I personally feel that Docker tries to do too much, almost the systemd of the container world. I believe having alternative container runtimes and build systems decoupled from Docker (both in the running program sense but also the company) will be the best in the long run.

With or without docker your workflow will remain the same. Its the image itself (the CRI spec) that makes that cross-platform magic work. I myself do my development on a Windows machine, ship a tar.gz off to Google's Cloud Builder to build the image and publish to a registry which then gets tested and debugged on a linux host.

Docker and cgroups/namespaces really only isolates you from mixed shared libraries situations, by essentially pinning your lib in the Docker image. That is, by ignoring the problem of stale/insecure shared libs, which is the entire point of using shared libs in the first place.

Docker doesn't isolate you from resource exhaustion (out of memory or files, infinite loops), from incompatibilities of the host kernel and Docker version bumps (so your shiny image isn't guaranteed to work on newer kernels and Docker versions), and makes it impossible to use host user identities and permissions. Thus projects tend to avoid plain regular file access, using databases and block devices and what not as workarounds.

IMHO Docker is an anti-pattern to "solve" an incidental problem of your own making.

You seem to imply that k8s somehow means having all your hosting and deployment provided by Google, which seems pretty fallacious.

Insofar as it gets people to use GCP, it's not necessarily very useful in that regard. Other replies have pointed out that k8s provides a useful abstraction above multiple cloud hosting solution providers. I'd like to add several reasons.

1. Alternative solutions exist. For example, Docker Swarm. It seems to me that Swarm, otoh, is too inflexible for mainstream popularity (?). 2. Alternative hosting solutions exist: why GCP and not, say, AWS? 3. Some companies build on top of the main k8s features and provide their own solutions tied to their own offerings, obviating the use of GCP. For example, rkt and OpenShift.

So even if Google is the dominant interest to the extent that k8s would fail if it stopped its contributions to the project, it is not obvious that Google will be able to capture all the value that comes out of the k8s project (of course, it expects to capture enough of it to make fiscal sense)

Could you expound on that a bit? I'm kind of new to the whole container thing and have been making an effort to educate myself, but it's not clear to me what's "wrong" with Docker Swarm that Kubernetes is so much more popular, especially when it seems like everyone is using Kubernetes with Docker already.

Alright, I haven't really delved deep into Swarm and this reply is based on ~15 mins Googling but...

1. Kubernetes since last year has first-class support for role-based access control to elements in the cluster, whereas I can't find access control concepts in Swarm. 2. I have a Kubernetes cluster where the pods simply assume and communicate in HTTP. Within the network, both Kubernetes and Swarm encrypt internal network traffic with its own CA, IIRC. Nevertheless, Kubernetes has an Ingress concept, which allows me to translate external HTTP connections over TLS into the cluster into HTTP connections (and if load is rerouted, re-encrypted with the internal-use certs, of course). This enables my containers to be focused and agnostic about the whole certificate shabang.

Amazon just made EKS (Elastic container for Kubernetes Service) generally available, so yes, other cloud vendors are wise to it.

All major cloud providers have/will have managed kubernetes and they are free to use the name, therefore I doubt it could be regarded as direct marketing for GCP. I would expect many to consider GCP but I just don't perceive this as a backdoor way of Google getting your apps, given the broad buy-in by the cloud provider community.

Well yes, you can replace Google with like two or three other companies in my comment. At the end of the day I don't like the trend of outsourcing all the interesting bits of technology to a handful of mega corporations. I think it is not a good bargaining position for us developers, Google and the like can already get away with paying shitty salaries and it will only get worse the more control they have. For all the smaller companies, they are paying a cloud provider instead of hiring local talent to build solutions. It's loose/loose.

That's effectively anti-globalization. Adapt and overcome. Good developers will always have a job in the forseeable decades to come, weak or stagnant developers will be at risk, it is a maturing industry. White knuckling tech by resisting change/momentum is no more effective than starting a trade war with tariffs.

Yes, there's a conformance/certification process required to use the name that ensures portability: https://www.cncf.io/certification/software-conformance/

Amazon is the 800 pound gorilla in the cloud.

Do you really want to see them completely dominate the market?

Kubernetes levels the playing field, and brings competition and choice.

That is a very interesting counterpoint! I guess I agree with you if the choice has to be between {Google, Amazon} or "only Amazon". But I'm advocating for the third choice, where we continue (or go back) to running our own infrastructure.

People will host their own infrastructure if it's the better cost decision for them. No one is stopping a company or an individual from going the self hosted route.

EDIT: I guess you could advocate against Google, Amazon, Microsoft in general for anti monopoly purposes, but that's not the same as using a cloud provider. Granted, those three are the top three offerings.

But I don't want to run my own infrastructure. Infrastructure is very hard to do well and the problems it poses are not necessarily problems that developers want to tackle. It also requires dedicated resources and staff that many smaller orgs have difficulty maintaining.

I'm fairly sure that's the main reason why AWS and GCP are so popular, not some nefarious marketing conspiracy.

If running your own infrastructure is a strongly desired option, people will do the work to make it easier to host your own k8s cluster. I agree it's not there yet (I'm the primary person for our team running several self-hosted on-prem k8s clusters) but I can see work is already happening to make it easier.

If anyone can comment on "I can see work is already happening to make it easier.", I'd love to hear it.

MS is also investing heavily in Kubernetes and their offerings on Azure is actually pretty nice. I've run k8s and OpenShift on premise, I've managed 'own infrastructure' - and no, I don't want to continue/go back to that. You cannot do that well and at that price on small-scale. If you think you can, you're doing something wrong.

In my company one person is the sole administrator for a 50 nodes k8 on prem cluster with only one additional person for the night shift.

So no, not hard to run, as long as you have the knowledge how to.

Having one person only managing 50 kube nodes IS the definition of too hard to operate. Operating 50 linux servers with something like docker swarm would be a no-brainer.

don't believe you will get down-voted for this point of view.

running and most-important keeping a k8s cluster up-to-date is a major time investment. I think the important time is balancing the benefits you get from k8s with the time draining aspect of having to run the cluster.

also, as counter-argument to what you are saying: if I'm going to move to the Cloud, why leverage k8s at all? Most clouds have other abstractions that you can use when building if you don't mind being locked into a specific bigCo.

Use k8s exactly for the reason of either being federated across cloud providers or simply portable across cloud providers. You can't do that with vendor specific solutions e.g. elastic beanstalk

What keeps people highly coupled with could provider is not where their code is running but rather other "here only" solutions like custom DBs or infrastructure.

You can rewrite code deployment script for any provider in few days worth of work, but switching between other tech stack will be near impossible.

> What keeps people highly coupled with could provider is not where their code is running but rather other "here only" solutions like custom DBs or infrastructure.

I agree, and the advent of k8s adoption is one less thing that locks you into a provider.

> You can rewrite code deployment script for any provider in few days worth of work

I've directly experienced the counter example to this at a major SaaS company. Beyond methods you'd use for < 20 machines this just isn't true. Your orchestration of machine lifecycles will be tied up into the particular cloud no matter what kinds of abstractions you try to put in. Unless you start running on multiple clouds or you write to k8s instead of the cloud APIs, you will be locked in.

Except k8s doesnt federate

Of course they aren't doing it out of charity. They're doing it to prevent an AWS monopoly and to ensure that their offering is the preferred one.

And we'd have cause for concern if 1) Kubernetes becomes dominant and 2) Hasn't accumulated a critical mass of non-Google developers.

That's what happened to Android, and why Google is very successfully closing it up again.

But neither applies to Kubernetes. AWS is still dominant so Google is still incentivized to keep Kubernetes as open as possible.

But more importantly, Kubernetes is much more than a Google project now. Google could pull all of its developers off Kubernetes today and it would keep going just fine.

Every choice in life is a set of trade-offs. Each cloud provider (AWS, Oracle, IBM, Microsoft, Google) is likely to embrace some form of Kubernetes or Kubernetes-ish container orchestration. And each of them is likely to end up in a situation where there is friction if you want to switch away should build on top of their offerings.

But that's kind of to be expected. It reminds me of using .NET and ODBC/ADO and for so long there was this "benefit" that went like this, "But you can switch your underlying database anytime you want without code changes!" And yet that is rarely true at face value and furthermore, how many people are switching their underlying databases from one to another on any sort of regular basis. Who cares about the benefit of being free from lock-in if you never switch anyways!

>After some time people will realize that running kubernetes yourself is actually harder and more fragile than just running your app without it

Wow you've made some really good points backed by examples and data.

In a world where everyone's on AWS or Azure or Google Cloud Engine, hasn't the ship kind of sailed?

Thanks for saying this man. Will keep this in my mind next time I am tempted to defend it.

You are correct, but... Would an AWS monopoly be any better?

Upvoted. Google is like a black hole, gravity pulls you in. Imagine Google disappears tomorrow, what then? Today we have for containers ECS, for email iCloud, for search Bing. Tomorrow? ...

And check Chromium fate. Hard to run it without Google services. I expect it to happen to Kubernetes too, unless we fork it now.