Hacker News Reader

Comment by manigandham | Hacker News Reader

Kubernetes already has many vendors that provide packaged distributions and long-term support if you want it.

If you use the managed hosting/cloud providers then updates are also automatic and easy.

I definitely have not found updates on GKE to be easy, even though all it takes to trigger it is clicking a button.

Each one I’ve done on our cluster over the past couple of years has caused something to break, often in a strange and obscure manner.

One time an automatic master upgrade happened at 9pm and effectively sent all of our logs into a black hole for reasons unknown. As with many of the other upgrade hiccups, I just had to poke, prod, delete and recreate entities and things magically started working again.

There is so much going on under the hood it makes sense that there would be issues with upgrades, so the frequency of the releases is frustrating. Dealing with after hours alarms just because the k8s release cycle is so aggressive really grinds my gears. I would fully support an LTS version.

this is the hardest thing for people to get sometimes, it was the original core "concept" behind devops/ci-cd:

upgrades don't get easier when you delay them and make them bigger, they get harder. if you think k8s upgrades are bad now, having them change twice as much half as often would be worse.

We've certainly embraced the devops approach, releasing many times per day with lots of automation on k8s, but I don't see this as being incompatible with using LTS releases.

Having an LTS release available offers the flexibility to choose when to upgrade based on any number of factors affecting the business and engineering cycles, the same way we pin software libraries to specific versions so our own stability and build consistency isn't subject to the varying release cycles of the package authors.

There's definitely a balance to be found between waiting so long to upgrade that it becomes insurmountable, and having to frequently run disruptive, mystery-downtime-causing upgrades for unneeded new features.

If you have an automated test suite for your applications, and you use it to ensure that candidates pass before deployment, why can't you test new Kubernetes releases by ensuring that your apps' test suites (which are known to be good) pass on new Kubernetes versions? If the test suite fails on a new Kubernetes version, then either improve the test suite to get rid of false positives or fix the software.

In proper CI/CD practices, build "stability" is an anachronism. Master is always green. Test changes before merging. If the changes fail the test suite then don't merge and don't deploy until it's fixed. Master stays green. Why is dynamic infrastructure any different?

LTS only makes sense as a strategy if testing and carrying out the upgrade involve significant amounts of work which needs to be planned out in advance. Ideally, this isn't the case.

That's mostly on the GKE team and not K8S itself. They run plenty of other extra components that arent well tested and often have issues during upgrades, along with the recent infrastructure updates for regional clusters, network endpoint groups and the rest that complicate things further.

Errors have never been major for us though, and they do seem to be greatly reduced recently with no noticeable issues in the 1.10x line.

I happen to be one of those vendors. I don't really view it as a good thing though. It's not good if everybody has to buy different forks of Kubernetes from vendors.

A point of reference, Amazon EKS only runs 1.10 which will not be supported by the community in 3 months. So not all managed services are really keeping up.

Ah, you're from Rancher. Great product, but isn't this exactly the problem space you serve?

How many of the K8S vendors are selling distributions vs support plans? Or are they always entangled? And if the community did do LTS, isn't that just shifting the work to open-source maintainers rather than focused and paid vendors?

Yeah having forked k8s will be bad. Back-porting possible security fixes could be tricky and having multiple companies have to do it, would be painful.

For on premises Kubernetes who do you know of who will provide committments to support k8s after the main project stops?

As far as I was aware of all the on-prem options cleave to the main project support lifecycle...

https://www.gravitational.com/telekube ?

yeah they mention some LTS for "Upstream components" from what I can see on the page, but not specifically which and for how long, I guess they might but it'd be good to see more info...

Disclaimer: I work for Red Hat Consulting on OpenShift.

Red Hat provides support our Kubernetes distribution OpenShift (upstream is OKD [formerly called Origin] project). We have an LTS release being cut soon as well. Watch OpenShift Commons for more info.

Feedback for OpenShift: Fix your support licensing terms and tooling. Having to manually deregister nodes on a web form to avoid hitting support limits is archaic and absurd.

I realize that the aim is mostly at the enterprise market who don't run autoscaling groups or cattle-grade servers, but you're scaring off potential clients with these kinds of setups.

I hear ya. I recently worked with a customer using ASGs and the scale down part of the ASG does need to make subscription-manager calls to avoid leaking subs. Also makes blue-green cluster upgrades more interesting as well.

If you have a specific suggestion/RFE, I’d be happy to push on it.

First, note that charging per CPU makes clients less likely to keep sufficient overhead to account for node loss or service scaling. Anything that makes availability a major financial decision is a bad thing (and openshift support is a relatively major financial line item).

A technical solution: A service that counts nodes or available VCPUs and reports it for billing. Kubernetes metrics makes this really simple.

A contract solution: Realize that bare metal is not the future (even for enterprises), and adopt a different billing model; following Oracle's per-CPU model is not the the way to be a leader in this industry. Per cluster, by bands of service counts, per site, per hour with minimum hour counts, etc.

bare metal is not the future, but very much a big chunk of the market, and it'll be for a long time. (and, I think, bare metal with k8s-like orchestrators is the short term future, and k8s + FaaS will probably bring the long awaited portability, that might help bring the hosting/infra prices down.)

that said, per core pricing is bullshit. and for exactly the reasons you mentioned.

That's cool :) Am I right in saying that'll be the first LTS OpenShift release?

What's the support lifecycle going to be on that?

Yes, it'll be our first official LTS release. I think the specifics aren't yet released but I think the intent is to make it more like the RHEL LTS policy & lifecyle.

Reply on news.ycombinator.com