I think a lot of Kubernetes hate is misplaced. It is a great piece of software engineering, well supported and runs everywhere. You certainly don't always need it but don't create a bunch of random bash scripts running all of the place instead of learning how to use it.
You'd need like 10 more Ciliums to make Kubernetes actually good, and they'll never all get adopted. The tech is a dead end that demonstrates some nice tech/patterns but got overhyped. It's like the Hadoop of containers
I like to think of kubernetes as a big orchestration platform that you can choose to use what you need. If an ingress and pods work then use that, otherwise extend an throw an operator up for what you need (it likely already exists).
Cilium for instance is great for that, so is Istio and the like. They aren't hard you just have to understand networking... which is nearly the same energy of running it on another orchestration tool or raw on a network device.
IMO you are thinking too much like an engineer if you are saying you “just have to understand networking”. Why? It’s always better when the problem gets solved in a way that allows you to not think of it too much. Right now SREs and the platform team do that for application developers because Kubernetes only does it halfway.
Infrastructure/devops/SRE is a pure means to an end of getting actual applications (the ultimate source of all the value in software) to run. It’s an obstacle. Right now the obstacle is bigger than it needs to be
Obstacle is big most of time because most applications are not easy to run. Most DevOps mostly came around because Devs flinging balls of mud over the wall and landing on the Ops side with a splat and then screaming when we can't build nets to catch the mud ball and Ops is covered with mud. Sure it failed just like DevSecOps fails because most of time, Devs don't care about anything other than closing Jira tickets and going home.
But companies don't want that, they want to support EKS and data centers, which means supporting the "implementation" side of all of the interfaces, which means getting down into the details.
The real problem is that every platform team, deep down, wants to rewrite EKS and they often do, which I would describe as "a giant money pit".
That will never happen, the only thing you can do is pay someone like heroku to take care of that for you. Or if your project is small and plain enough you run “serverless” which is just routing to another platform team, as I’m sure you’re familiar.
It’s complicated because there’s a lot that goes on, kubernetes or not.
In other words - Serverless is another platform team in the same way Linux is another OS team or Java is another language team. And for 99.9% of companies a language team or OS team would be absurd. There’s a big incentive for platform work to go the same way.
Things like having to remove finalizers appended onto resources when you delete them and they hang.
Damn near everything about Custom Resources.
This isn't a dig at Kubernetes; I love using it. However, I agree that it leaks implementation everywhere.
Most people in industry have never used Borg or something like it, and so think Kubernetes is like apex technology rather than a step backwards. And to most people working directly on tech like Linux/containers/low-level infra is something they don’t even consider doing because they subconsciously think it’s off limits. So to them Kubernetes is amazing rather than riddled with fixable problems.
The best part is multiple environments. I can run our full stack on my laptop with k3s and one call to make. We use the same yaml (with kustomize) for all three cloud environments.
It feels pretty boring, which is fantastic.
And the fact that many medium companies have “platform teams” configuring Kubernetes and gluing together basically the same set of tools (source, build, test, release, ops, obs) together in basically the same way is a huge smell that something better is needed. Basically, doing things the right way is actually a big operational/engineering/monetary burden for most companies that just want to write applications. And K8s is a big part of that
I do agree it could be better but it's Un opinioned nature also let's these companies shape it in a way that matches where they are. Also, probably biggest issues with it only come up if you are not in managed Kubernetes which many are.
The whole ecosystem around it is an example of Conways law and and a Google product.
None of the people using it are google.
Google, also, runs its own hardware.
Shockingly it is a great product if you rent hardware, autoscaling is autospending. No one knows what a feature costs any more because its all just a big bucket your pouring money into for amazon to have 30 percent margin on.
We need operations people again, we need to stop using containers as bags for shitty "software" ... Do actual engineering.
Have worked with numerous cloud native engineers that do not have good foundational knowledge.
That's the problem. Yes, I'm the old man yelling at cloud, but indeed 'kids this days' don't know Linux, they know how to provision things from Terraform. In fact it feels like logging in via ssh and checking process with with `strace` is a lost art. Checking PCAP? That's a black magic!
No, I want to hire the grumpy, cynical greybeards, because they know their fundamentals, and they're immune to shiny bullshit.
There's no substitute for skill and experience, despite what the modern tech discourse says. "Anyone can program!" And yet, we have threads like this, where we find out, SHOCKER, that the best codebases are the ones maintained by like... four senior developers, half of whom contributed to several Internet RFCs, and half of whom used to work at Bell Labs.
I feel like we are through the looking glass.
You need to include checks when writing them so they actually are idempotent, just like with bash script, and they happily leak idempotency-breaking details like "restarting a service" works only if a) service currently runs b) because "restart" is a command, not expected state of "the service was restarted and is now running
I’m told this is gatekeeping. Yes, and…? I’ve always found it amusing that I see little to no anger being directed at kernel devs for their gatekeeping. Almost as if deep down, people know that they shouldn’t be fucking with the thing that runs the world unless they are actually good at it. Or maybe they’re just afraid of Torvalds.
It’s been a long time since I’ve had to do those things, so yes it’s becoming a lost art to me. However, on the rare occasion I need this sort of insight I do fine with JIT research.
Still in the same place they were 20 years ago. They’ve just been lost in the swamp of their peers that know nothing.
A lot of big dinosaur corporations are implementing it actively. Unfortunately VMs or Kubernetes or whatever tooling is still going to suck if you have shitty people using them.
This is such an inaccurate take.
>we need to stop using containers as bags
Containers and container orchestration are a NEEDED and REQUIRED piece of the technology stack in the current reality. That doesn't mean people need to be ignorant of the details that make them work.
As someone who's been around for a while we are in a better place than when we had "dedicated operations people" that just gatekept everything.
What you hate are teams just parroting what other people are doing with the tech, not the tech itself. But let me tell you, if you're going to have teams of people pretend to know what they're doing wrapping it in a standard "bag" sure does make it a hell of a lot fucking easier to unfuck when things go wrong.
If you're deploying NODEjs apps to the cloud, they sure are.
> if you're going to have teams of people pretend to know what they're doing wrapping it in a standard "bag" sure does make it a hell of a lot fucking easier to unfuck when things go wrong.
No one looks in the bag, they just roll back, the bag is opaque, the bag breeds more bags, less trnasparency and higher costs.
Step back and read what you wrote, if you described any other relationship in your life in these terms, your friends would be having an intervention. The codependent circle jerk has enabled so much bad behavior. The tooling you're advocating for is an enabler.
I want App Store like software roll outs. The infrastructure we have is kind of like that, if you don't look too close. We need to start doing the hard work to make it not suck. We need to stop pretending that this is a good place and put in the work to make it better.
Otherwise its never going to get untucked...
I don't use nodejs.
>No one looks in the bag
I sure have. Plenty of times.
>if you described any other relationship in your life in these terms, your friends would be having an intervention.
For having a standard tool that you know how to use? That doesn't seem correct and it certainly isn't proportional.
>I want App Store like software roll outs.
Okay it sounds like you want K8's.
That is not remotely true. Containers (and their orchestration) are a choice. Obviously people have their reasons for making that choice, but it is in no way a requirement. To say otherwise is either profound hyperbole or ignorance.