Hacker News Reader

Comment by weitendorf | Hacker News Reader

It's basically a prototype that industry ran away with. It leaks implementation details everywhere and pushes way too many config options up to the developer. Because industry ran away with it before it could be good, it takes projects like Cilium to push it in the right direction, but those take forever to get adopted and are really hard because they don't live in the product itself.

You'd need like 10 more Ciliums to make Kubernetes actually good, and they'll never all get adopted. The tech is a dead end that demonstrates some nice tech/patterns but got overhyped. It's like the Hadoop of containers

View on HN

Can you clarify what you mean by "leaks implementation details everywhere"?

I like to think of kubernetes as a big orchestration platform that you can choose to use what you need. If an ingress and pods work then use that, otherwise extend an throw an operator up for what you need (it likely already exists).

Cilium for instance is great for that, so is Istio and the like. They aren't hard you just have to understand networking... which is nearly the same energy of running it on another orchestration tool or raw on a network device.

You shouldn’t have to think about all the implementation details of your deployment target. There shouldn’t be platform engineers or Kubernetes experts. Nobody should be writing YAML, or getting paid to set up Istio. Nobody should have to learn the Kubernetes architecture or know about the kubelet or EBPF. The tools should be simple enough with good defaults that application developers just click a button and have their code run somewhere. Right now platform engineers fill that gap, because the underlying tech doesn’t.

IMO you are thinking too much like an engineer if you are saying you “just have to understand networking”. Why? It’s always better when the problem gets solved in a way that allows you to not think of it too much. Right now SREs and the platform team do that for application developers because Kubernetes only does it halfway.

Infrastructure/devops/SRE is a pure means to an end of getting actual applications (the ultimate source of all the value in software) to run. It’s an obstacle. Right now the obstacle is bigger than it needs to be

So there should be magic on every layer except for the application?

That will never happen, the only thing you can do is pay someone like heroku to take care of that for you. Or if your project is small and plain enough you run “serverless” which is just routing to another platform team, as I’m sure you’re familiar.

It’s complicated because there’s a lot that goes on, kubernetes or not.

You literally use magic like that all the time with Linux, compilers, language runtimes. Magic is perfectly good when it works. Being able to do anything you want with a quick “presto” is amazing. Nobody should have to learn arcane spells, study the ancient tomes, and communicate with beings of the ethereal plane - somebody just needs to figure out the next “presto”. And historically speaking, usually somebody does when there’s an incentive to do so.

In other words - Serverless is another platform team in the same way Linux is another OS team or Java is another language team. And for 99.9% of companies a language team or OS team would be absurd. There’s a big incentive for platform work to go the same way.

Ops here, In a simple Kubernetes environment, you don't have to know networking. However, few environments are simple and abstracting X away becomes extremely difficult job once business requirements collide with abstraction.

Obstacle is big most of time because most applications are not easy to run. Most DevOps mostly came around because Devs flinging balls of mud over the wall and landing on the Ops side with a splat and then screaming when we can't build nets to catch the mud ball and Ops is covered with mud. Sure it failed just like DevSecOps fails because most of time, Devs don't care about anything other than closing Jira tickets and going home.

I think that devs really should have a decent understanding of Kubernetes. It is essentially the operating system for any app that needs more than one computer.

You don't need any of that, EKS works out of the box with Fargate.

But companies don't want that, they want to support EKS and data centers, which means supporting the "implementation" side of all of the interfaces, which means getting down into the details.

The real problem is that every platform team, deep down, wants to rewrite EKS and they often do, which I would describe as "a giant money pit".

Is it fair to say that you are in favor of Heroku, fly.io, ECS, etc?

I think those products are much better than k8s for many use cases, but I don’t think that anybody has really delivered the right thing in that space yet.

Just look at the release notes for every major release. They focus on what's new within Kubernetes architecture instead of what's new that benefits users.

Things like having to remove finalizers appended onto resources when you delete them and they hang.

Damn near everything about Custom Resources.

This isn't a dig at Kubernetes; I love using it. However, I agree that it leaks implementation everywhere.

Kubernetes is just a poor Linux clone with extra steps. Seriously, it has all the basic parts of an OS, just half-assed: the scheduler, the networking, the state management. We already had way better operating systems that can, you know, schedule workloads and talk to the network, and it didn't require gigabytes of YAML and string templating to make it happen.

Yes. I actually wrote another top line comment about this which I deleted. Kubernetes is like someone decided to create a distributed Linux where you had to configure and learn about every part of the operating system. Linux proves it’s possible to provide simple “deep interfaces” to OS-class, hard problems.

Most people in industry have never used Borg or something like it, and so think Kubernetes is like apex technology rather than a step backwards. And to most people working directly on tech like Linux/containers/low-level infra is something they don’t even consider doing because they subconsciously think it’s off limits. So to them Kubernetes is amazing rather than riddled with fixable problems.

Reply on news.ycombinator.com