Hacker News Reader

Comment by goalonetwo | Hacker News Reader

I read the whole thing and I'm not sure I understand what IS the point except that things are too complex? A whole lot of words for not a lot of content.

View on HN

Fear of complexity is the number one reason why many "simple" homebrew solutions were born to solve simple problems at time, from what I observed in recent years across a few companies I worked for. The hard truth is, fewer simple things can stay simple while continue being useful in the long run.

The value I get from Kubernetes, is the API platform that allows my system to organically become more complex overtime, but hopefully in an organized way.

Yeah, the way I read it, it boils down to "I worked on Docker. Kubernetes is great but too complex for developers to actually use." Which yes, is something that everyone who isn't fooling themselves has been saying since its inception. I've been managing on prem kubernetes clusters for the better part of a decade, a lot of it makes sense to me now, but was absolutely painful to learn. In my opinion the article would be better if it had a clear goal for the future, but I couldn't find one.

Yeah exactly.

There is no good solution here as in order to have a SIMPLE platform, you need to really reduce the amount of use-case you support to a couple options only. The reality is that every single company I have worked for has a ton of different services with different requirements. Those choices end up being reflected on the platform that now needs to be able to run all of them. If you have a single simple use-case then you should just run that simple use-case on a boring EC2 instance with a boring dockerd and a boring NLB in front (as an example) but the reality is that this is almost never the case.

If you have a lot of different workloads to run at scale there is no better tool than Kubernetes. The complexity comes from its power to run almost anything you want

Kubernetes is a generalized solution to the problem - hence the complexity. Tools such as Docker Swarm were tailor fit to solve a particular application deployment and scaling problem - hence the simplicity.

I haven't looked in a while, but I recall at one time there being a utility that could take a Docker Swarm yaml and convert it to Kubernetes - Kompose? Has that matured into something useful?

On the subject of Kompose, frankly, I think a person looking to get into Kubernetes either knows how to convert what is essentially a docker-compose file to kubernetes manifests themselves, or they're going to struggle working with kubernetes to the point that no amount of "this is how you convert to this" tooling will save them. At the end of the day, you're working on kubernetes, so you need to know how to work with kubernetes. That might come off as gatekeeping to some, but it is an honest estimation of how much effort goes into productionizing workloads for kubernetes.

That's fair. It's usually worthwhile to suck it up and learn the basics of what it is you need to do.

Exactly. And I did a poor job of mentioning this in my previous message. But that perspective comes from trying to rely on Kompose to migrate my apps over to Kubernetes for me, and then falling utterly short in every aspect that makes Kubernetes inherently more powerful but also more complex than Swarm, such as in the domains of networking and storage. I ended up having to circle back to Kubernetes learning courses on Udemy, setting up a playground, making tons of mistakes, and accidentally becoming more proficient in Kubernetes than I had anticipated, just to get decent at deploying things to Kubernetes. And to be quite honest, I'm still learning about 6 years later. Lately less about storage and networking, and more with securing my workloads using securityContexts, runtime image scanners, etc etc. I'm at a point where I've written my own and helped contribute to Kubernetes operators, but I'm still learning how to effectively use all the tools Kubernetes gives you. So this took a bit of a tangent from the topic of Kompose, but it's all to basically say, "you'd be doing yourself a disservice to blindly rely on Kompose to get you into Kubernetes." In my opinion, of course.

Disclosure I work for a company, Noop [1], which is the picture of the steak.

As someone working on the marketing side of the endeavor, I see the shift the author describes. there are a handful of PaaS companies picking up where Heroku left off and in the enterprise world devops is evolving toward "platform engineering". Platform engineering suffers from being poorly defined, but there appears to be growing demand within large enterprises for something like internal-heroku. But there's still a problem.

To me, the problem is not kubernetes. The problem is that tooling has become so specialized that the focus of work has become integration between tools. And that cumulative integration work complicates the operational responsibilities of software developers. Even if you have a dedicated devops team, the complexity of those integrations flows down to developers in the form of different systems for logging, monitoring, firewall, cdn, ci/cd, secrets, etc.

1. https://noop.dev

My take is DevOps and SRE didn't take off how big companies thought they would and the outcome is platform engineering which used to be more a combination of infrastructure engineering and sysadmin roles

Noop actually looks cool, I’ll give it a spin sometime. I’m using Fly.io for all my stuff at the moment but recognize at some point my infra my outgrow it…

1. Developers want to deploy their apps easily, 2. The default is to use containers, but that's only a part, 3. Because of containers, k8s got very popular, 4. But for developers, it doesn't really matter what the underlying tehc is, it could be k8s, it could be k8s, it could be anything as long as it works smoothly, 5. that's called IDP or PE now, but it's similar to what Heroku offered years ago

The point was that too many firms focus on Kubernetes as a thing in and of itself instead of focusing on app development.

From a business perspective it can, and most of the time does, benefit an organization to outsource app deployment.

I’ve seen first-hand organizations that have gigantic teams of DevOps working endlessly on perfecting these systems, all the while taking teeth pulled for an app developer to get a new container deployed.

It's possible that those platform engineers did a poor job exposing kubernetes to the developers, in the anecdotes you've mentioned. But there is a LOT more being instrumented in an effective deployment platform than just deploying an app somewhere. Centralized logging and metric aggregation and then visualization of those is an important aspect to any platform for app deployments that get overlooked by many naive developers. Deploying a service to some PaaS site is great until you need to extract any runtime information out of it, if it doesn't also instrument those features, which I've been a consumer of many of them that don't really do that at all. And then performant and reliable shared cluster storage is another animal altogether in on premise kubernetes clusters in particular. So far my best experience in that area has been with Ceph, which is like learning a completely different distributed system with its own complexities.

The app developer doesn't see it, but behind every decent app deployment in kubernetes, there are millions of hours of expertise in running distributed applications reliably. And the simpler deployment platforms do a good job of abstracting some of this, but from what I've seen nobody does all of that exceptionally well, and if they did, they could probably be able to charge whatever they wanted to people wanting to use it.

When webmail (Gmail, Hotmail, etc) first came out, one objection I remember is that it was a bad idea, because it meant you couldn't get at your email if the service went down. How foolish of you to rely on someone else to keep their website up 100% of the time. That thinking passed as expertise in keeping systems up became its own branch of software engineering. So we forget that it's very possible to do a bad job of keeping a website up. some of the platform engineering work isn't just for show, it turns out.

You need about 10% of that functionality to reliably and profitably sell screen real estate to advertisers.

Reply on news.ycombinator.com