thockingoog

This is explicitly part of the process of transferring each property. We need to be able to produce reports on spend in each area, and those reports are property of CNCF, not Google.

Well, the headline is pretty awful. It's kind of personally offensive to see what is really, truly a well-intentioned act get spun in nasty ways.

Besides that, 2 people does not an army make...

We gave this project to CNCF years ago, and Google is still pretty heavily invested in Kubernetes. :)

This is the end of the last vestige of exclusive control.

Run 5000 VMs more or less continuously. Spin up/down hundreds of kube clusters every day - on every GitHub Pull Request. Serve 9-digits worth of container images per month. To name a few.

The infra behind k8s is kind of staggering.

That's hyperbolic. Part of the announcement is a commitment to re-evaluate the numbers as time goes on.

Google is in No Way stepping back from this.

Disclosure: I worked on this grant.

TL;DR is that CNCF owns kubernetes, but until now I could not include non-Google people in the administration of (ostensibly) community owned stuff (google.com security policies in GCP). Now I can.

Google's involvement and commitment is in no way diminished, this is just a bad headline.

Kubeadm and others help install, but they don't manage clusters over time. We find that the ongoing management of GKE is what really speaks to people.

The workloads will still be as portable as ever, of course.

Perhaps not surprisingly, this is where Istio excels..

I feel like I have to say this, even if people get it already.

Walk before you run.

Bare metal is a LOT harder to manage because, well, hardware fails. We hear the demand, for sure, but vSphere represents walking (and has a lot of customers, too :)

The mechanism will probably support it. Whether that is "supported" or not I think is TBD. :)

Deploying containers is shell-script-easy as long as nothing goes wrong. Ever. I know you know this. I'm a little confused why you'd make a claim like this.

I serve a couple small websites from a very very small GKE cluster that costs well under $100/month. I don't worry about OS updates or Kubernetes upgrades. I don't worry about it going down.

I don't think my needs are that unique.

...and as of today there is no fee for masters on GKE.

https://cloudplatform.googleblog.com/2017/11/Cutting-Cluster...

First, Ingress had a purpose to serve, and it has served that purpose - it is relatively easy to handle low-complexity apps with generic Ingress.

But low-complexity apps don't stay that way.

What you're describing is very much the way my brain has gone. In my experience, most users end up using at least one non-portable annotation on Ingress. The logical conclusion then, is that people care about features MORE than portability in this facet of the API.

This is not surprising to me, given how religious the debate tends to be...

Google and GKE team member here: The AKS announcement was either poorly worded or intentionally vague. I choose to grant the benefit of the doubt.

For small clusters (less than 1-5 nodes) GKE does not currently charge anything "extra" except your own node VMs. That is LITERALLY a $0 master.

Beyond 5 nodes we currently charge a flat rate that covers your zonal master(s) regardless of how many or how big they need to be.

The problem with Ingress "lagging" is that it was designed to be a lowest-common-denominator API - it only absorbs logic that exists in the majority of realistic implementations. For better or worse, cloud LBs are vastly more limited in feature-set than Nginx or Envoy, so Ingress is too.

This is a big topic for debate, and will be on the agenda at KubeCon in O(days).

Are people happy with a built-in L7LB? I find most people are very particular about which LB they use.

The link you posted highlights my exact concern with Swarm. If I use port 8080, then nobody else can use port 8080. That is a different tradeoff than Kubernetes is willing to make.

Functionally, we have this in NodePort, but because it is exclusively managed, you're very unlikely to have a meaningful conflict. BUT it depends on traffic ingress being managed.

If you just want to map port 8080 on every node into your kube Service, it's more complicated than Swarm. Granted. That's because we don't think you should do that - it doesn't scale.

If you just want to map a port on every node (and you don't care which port), kube Services have you covered. Swarm's model is a nearly-direct clone of this.

> Docker Swarm secrets have been GA for longer thank k8s

I don't think that's true. Kube secrets were introduced 2015-02-17 and was considered GA in Kubernetes v1.0

> I think the pluggability of critical pieces like ingress and secrets was taken too far.

I think the pluggability is not the concern but the lack of an included solution. Part of the problem is that SOME platforms have an included solution - e.g. Google Cloud, and some need 3rd party code like nginx.

> I dont think that is true, it does manage its overlay networks pretty well

Overlays are a waste for most people. I get that making it simple is attractive, but it's (IMO) not something everyone wants or needs. Again, we could/should have had a built-in option.

Last I looked (admittedly a while ago) Swarm had a pretty deeply rooted notion of exposing ports on all nodes in the swarm, which means that if you have multiple containers that need to expose the same port, it was a problem. Kube takes extra complexity here, to make it possible to share arbitrarily.

Anyway, it's not my intent to bad-mouth Swarm or try to convince you that you're wrong. Different trade-offs were chosen for the two systems. Your feedback is noted and appreciated. :)

This is an apt point. Kubernetes models Borg, and Borg has no concept of ingress. That's an entirely different problem space.

Obviously that doesn't fly if there isn't an equivalent open solution, so we did what we could with the system to make it not terrible. We can do more.

The point about Swarm is interesting, and has been much on my mind. Some of Kubernetes' perceived complexity is because we go to great lengths to avoid ever having two users collide, with escape hatches for the people who really need "unfriendly" features. This is because, again, Kubernetes models Borg. Borg clusters are giant, shared, multi-user, multi-app animals, where the users are in different business units and chances of collisions are high.

Swarm, on the other hand, thinks of a cluster more as an application construct. Sharing is not a big problem, and coordination is easy and local. This allows them to make different tradeoffs. I doubt very much that you can run a large number of similar apps in a single swarm without having collisions on things like ports.

I still believe the large-shared-cluster model is right in the limit. There are so many efficiencies to be had. But there are legit reasons it is hard to achieve right now.

I'm very interested in ways to make Kubernetes easier to use, ESPECIALLY in this regard. Real user feedback is critical.

I hear what you're saying, but don't under-estimate the value of "someone else runs it and gives me an SLA", which is a large part of what public clouds are really selling. :)

A great many of those (but not all, I admit) have equivalents on Google Cloud. I know I am somewhat biased (as I work there) but the technology is really great. BigQuery is untouchable. Spanner is magical. CloudML is hands-down the best offering. Etc.

Have you ever considered "what if I didn't presuppose AWS?"

I don;t think that is really what happened. :) As part of the runtime abstraction, kubernetes must necessarily decide how to handle stdout/stderr logs. Not having a decision for something at a given point in time is not exactly a bad thing.

Please remember that Kubernetes itself is literally less than 2 years GA. Stateful is hard. It wasn't the first thing we tackled. It's in progress now, and feedback so far is pretty good.

That may not be enough for you yet, and that's OK. Check back in 6 months and see how it is progressing.

Hey man, I get that you need something NOW and I am sorry about that, but I have to say this is a teeeeensy bit over the top.

Yeah, StatefulSet is still beta. We're getting miles on it before we tell people that we 100% back it. But you know what? People ARE using it. In production. With real data. And they mostly are just fine.

I run a (tiny) database against k8s. I trust it with my own data.

What you say "the container engine underlying k8s will delete all changes made to the image" is true - don't write files straight to your image FS! This is containers 101. We have PersistentVolumes for this very reason. Data that has a lifetime of its own.

Does this absolve you from backups? No. Does it mean you don't need to think about upgrades? Hell no, you still have to know what your apps are up to.

Nobody is making people use containers for databases, but the power of systems like Kubernetes is pretty addictive, and a lot of people are pouring a lot of energy into this problem.

Is your jenkins code public?

> come back to Kubernetes once its release cycle is a bit saner for production environments

Out of curiosity, what sort of cadence would you like to see?

ACK. We recognize this aspect of the docs (engineers writing docs == encyclopedia articles). We have real, honest to goodness tech writers now, and they are working on it. One piece at a time.

I think Solomon has hit the nail on the head here, but I'll call it out even more starkly.

Assuming containerd is successful, and that higher-order systems like Kubernetes and Mesos use containerd directly, we will see one of two things in 12-18 months time: 1) There are hundreds of thousands of DOCKER users 2) There are hundreds of thousands of CONTAINERD users

The split is forcing stratification (in a good way) where there previously was none. Users have to identify whether they think "docker" means "a container runtime" or "a full stack".

Of course, the pie is growing, so maybe we get both results. The real problems over the next 12-18 months are MESSAGING and DISENTANGLING. How do you get this message across to people, and can you actually change the words in the common vernacular?

As owners of the word "Docker" Solomon can define it how he likes, but that doesn't mean he can actually stop people from using it to mean something else. That's going to be a process.

c.f. "literally". Even Webster's Dictionary has given up the fight on that.

THAT is true. Kubernetes on its own is not a product, per se. There is no one company behind it. No legally binding support contract. etc.

It is the basis for many products (plural) and an ecosystem, which is really what we wanted to achieve.