Hacker News Reader

Comment by wayoutthere | Hacker News Reader

I mean, good on the author, but this isn’t what Kubernetes is really for.

Kubernetes is basically a way to run a Java-like application server that can run things other than Java. If that sounds like an appealing prospect to you, the complexity of Kubernetes may be a good fit.

Kubernetes is complex because sometimes you need to be able to do complex things. Sometimes you operate at a scale where spending 12 hours writing a deployment script is ok, because it will save you hundreds of hours in the near term. Kubernetes expects you to write a bunch of custom integrations to tie your k8s clusters into whatever ITSM / ITIL process you use.

But complaining that running a blog on Kubernetes is too complex is like complaining that a semi is a terrible vehicle because it’s hard to park at the grocery store.

View on HN

Kubernetes is the new Java Application Server for people who didn't realize that Java Applicaiton Servers were a terrible idea.

Despite a long track record of failure individuals are trying to introduce the complexity of J2EE onto kubernetes. It doesn't need to be that way. Kubernetes can be very simple and it has been up until recently. Once the Enterprise Architects got their hands on it and decided everything needs to be a plugin and nothing should work out of the box the complexity started to creep up.

You should be able to run your small blog on kubernetes without requiring a team of consultants to set it up or manage it. Just waving your hands and saying well it needs to be complex to scale is a total lie.

Odds are someone / some people will create some kind of simpler solution with an easy default setup within the next ~5 years. Maybe as a wrapper over Kubernetes, or maybe as something new and interoperable with it.

Maybe it'll involve a bunch of "serverless" buzzwords or some newly invented buzzwords. That's how things usually go historically. A lot of value can still be extracted if you're careful to ignore the cult-y bits. Containers, serverless/FaaS, and Kubernetes can be pretty great if you're the plane pilot dropping the cargo on the island rather than the cult living on it, and future stuff will probably be even better.

Oh they have and are; but what is the business case open source such a wrapper? Wouldn’t you just host it on your own hardware and basically be another Heroku? This kind of software is complex enough it would take corporate backing (either an industry group or VC backing) so it’s probably not getting built unless there’s a business case.

Even Google only released K8s because they thought it would push people towards GCP — which it didn’t because it was too easy for AWS to implement a similar service using Google’s open source tech. The open source freemium model for infrastructure tech is pretty much dead as a result of this kind of activity.

The wrapper can still be something self-hosted. There will always be SaaS/PaaS/IaaS abstractions out there for just about everything; even MySQL. The idea would be for someone to be able to very easily self-host something that's as simple to use and configure and interface with as AWS EKS or Google GKE.

I have no idea what business case there'd be to open source it. Maybe some open source devs with former experience at a big company will create one just for fun after they leave the company? Who knows. It'll probably happen eventually, by someone, either way.

Would you do your day job “just for fun” after you just quit?

The reality is that most of this technology — especially around infrastructure — has become so complex at scale that the tech strategy and the business strategy are the same thing. So infrastructure software has to match your business architecture which is largely dictated to a technology org. Which is why “tech ops” these days is largely just “ops” — the technological complexity is a reaction to increased business sophistication, not the other way around.

Me? No. But some people seem to like to do that. And some people do seem to be genuinely very passionate about infrastructure and such. Kubernetes can be applied to a lot of different business and technological architectures, I think, and a simpler alternative could be similarly general.

> it was too easy for AWS to implement a similar service using Google’s open source tech

AWS had the most K8S deployments even before they released EKS, according to the CNCF:

https://www.cncf.io/wp-content/uploads/2018/08/cncf_survey_g...

Or perhaps somebody will evolve Kubernetes itself into becoming simpler?

I know that's a pipe dream, but really, why does it have to be? What would have to happen for people to actually work on making existing things simpler and better factored rather than reinventing the wheel?

My personal theory is that it's largely because that kind of work simply isn't being valued highly enough. Reinventing the wheel is a much lower friction path to take and has a higher chance of being rewarded highly. It shouldn't be like that, though.

I think k3s is sort of trying to do that?

Have you tried it? Is it any good? I've been looking for a simpler Kubernetes, although I don't know how much simpler it can be in practice and still do the same things.

I've tried k3s, and while I was impressed, it leaves out several important features.

RKE is a great alternative, and kubespray is quite stable well.

How about Hashicorp nomad? (https://www.nomadproject.io/)

The problem with making things simpler is compatibility. If you make something simpler, but don't stay compatible, it might be better to just find a new name for your simple version.

Do we even need k8s for a personal blog? What problem does it solve for someone coming from docker or a VM?

k8s is a building block one can use to provide a simpler service, and if you want to convince anyone it needs a refactoring, maybe provide some specifics?

That would be knative.

> Odds are someone / some people will create some kind of simpler solution with an easy default setup within the next ~5 years.

I know it's not the same but Docker Swarm is pretty great if you just want to deploy some container images on a single host or a cluster - this guide covers setup + traefik + swarmpit ui https://dockerswarm.rocks

This is the cycle of life. Zawinski's Law is a powerful force.

The same thing happens with ticketing systems: $old_ticketing_system is way too complicated and bloated, so let's jump to $new_ticketing_system because it's small and easy to understand. Oh, but we miss $feature_1, so let's ask for that. And $feature_2, and $feature_3. Continue until $new_ticketing_system becomes way too complicated and bloated, at which point you find $newer_ticketing_system, which is great except that it's missing $feature_3. Oh, and $feature_4. And 1 and 2, come to think of it. Oh geez, now $newer_ticketing_system is also coming apart at the seams, time to migrate to $even_newer_ticketing_system. . .

This strikes as the exact opposite of reality. Java App Servers were specifically built for vertical scale. You just paid $80,000 to rack 30 CPUs and now you need to a way to optimally utilize all of them so we have a deployment model for sticking multiple applications in a single multi-threaded runtime. That was a pretty decent concept for 2005 and was pretty successful. The concept of packing code into archives (jar/war) has proven to be pretty durable.

Kubernetes is explicitly about managing horizontal scale where the hardware is abstracted away.

That's not the point of the analogy. The point is that k8 and JAS are designed for "large scale" deployment, whatever that means.

(But yes, you're correct about what "large scale" meant in the early 00's)

You don't think Kubernetes is used for bin packing?

I think you’re getting it backwards here. Kubernetes was explicitly built because the existing solutions were not robust enough to enable containerization at Google’s enterprise clients. Docker existed, and there were plenty of quick and easy ways to deploy your blog from a docker container and get it working. Those still work today.

The “running my blog” use case is a Docker use case. Kubernetes was designed from the ground up to enable transparent integration between containerized apps and ITSM platforms. I have always viewed it as more of a scaled application framework than a hosting platform.

But then the question arises, why would I need kubernetes to run a small blog... why not just run the blog. I mean.

What about Nomad? It looks simpler, although it doesn’t seem to have batteries included for things like ingress. I’ve never used it, but I’m curious.

The main problem is that smaller shops are adopting the fads of very large tech companies, but the large tech companies usually adopted those tools to deal with the kinds of scale that the smaller shops just don't have.

I’m not sure I agree. K8s’ primary benefit is that it strings together the IAAS abstraction into a single api.

It allows you to deploy multiple replicas, automatically setup a load balancer and handles maintaining the link between the LB and the backend. While also replacing any failed replicas.

Completely valid points, and I agree to all of them. Alas, as other commenters have pointed out, the issue is devs at smaller companies deploying smaller products buying into the idea that they need k8s. I believe that it is the community's duty to educate these devs on what k8s is and when it is needed.

They are probably scared that when they need to change jobs, the next company will require "5 years of Kubernetes experience". So they convince everyone at their current company to jump into a complexity clusterfuck to see how it works "in production" and can put it on their resume. This is how the entire IT industry works today.

I am fucking appalled writing config files is a noteworthy skill is 2019. So should you.

K8S is hardly just "writing config files" is it? You've still got to understand a ton of moving parts underneath before you're able to.

K8s is writing config files just like Python is writing Python Syntax.

If you don't understand the underlying mechanism, either with Python or K8s yaml files, you're going to have a very bad time.

Somewhat ironic side note - Asking folks to write K8s config files is exposing too much complexity for some developers I work with. And I kind of get it. Properly setting up a service with changing environment variables, secrets, ingress, API Roles, AWS IAM roles, and horizontal autoscaling can get a bit nuts.

Yeah, fully integrated “DevOps” at scale is a pipe dream. You will always have some segregation of dev and ops because the scope of knowledge is so different, especially today where “Ops” often means “expert in XXX cloud vendor’s product portfolio and how our operating model uses the features”.

What we call “DevOps” is really a delicate balance of giving the dev teams enough rope to hang themselves while child-proofing the gallows.

I don’t think DevOps leaders are claiming DevOps should be fully integrated so much as there should be a culture of collaboration and empathy, shared metrics and incentives, and preference for end to end automation... rather than antagonistic “throw it over the wall”, “I’m a dev and am too important to be paged” behaviour, etc., which has nothing to do with skill specialization.

Good contracts lead to good collaboration. Kubernetes provides the foundation for a solid end to end contract for managing complex systems automatically. It’s incomplete, but extendable.

> “I’m a dev and am too important to be paged”

Also sounds too important to be dev.

> What we call “DevOps” is really a delicate balance of giving the dev teams enough rope to hang themselves while child-proofing the gallows.

Oh, dear, @wayoutthere, my dev team is about to hate you because I'm going to use that quote extensively over the next few weeks... (giggle...)

You realize that you're commenting in a thread for a blog post that's 100% about configuration, right? There are articles like this popping up on the front page here every few days.

I've never said that K8S is just about writing config files. I've said that it is appalling that wiring config files is still a technical "skill" that warrants articles and discussions in 2019.

What's even more ridiculous is that most people here probably don't even see any alternatives. I've had this conversation several times and it inevitably reveals the unwavering (and irrational) belief that manually entering cryptic text somewhere is the only way to make reusable configurations.

Truly, we have become the tools of our tools.

I really hate the term "Configuration as code" because it's not code. For most of us, code means something that can be stepped through. For many, that means stepped through in a debugger. Descriptive languages have almost never provided that facility and we don't have it with Kubernetes or Docker Swarm.

Yes it's great to capture your configuration in version control. But if at the end of the day I'm staring at a config file in one window and a log file in another and waiting for enlightenment to grab me, that's not scalable and it's rigid. It also pisses me off to no end.

It's essentially the Frameworks vs Libraries debate all over again. I'd much rather have something imperative.

Declarative systems create a perverse incentive to keep things the way they are because it's difficult to reason about how changes affect the system, and it's virtually impossible to explore those effects. There are no guideposts you can use to apply Local Reasoning, and so there is no pressure to organize this 'code' in a manner that supports it. So as the system matures, everyone is working off of memorization. There are too few bite-sized chunks that can be learned a bit at a time. You are locked into your current way of thinking and you've locked out anyone who can bring fresh perspective.

It doesn't take a genius to see this will end badly. Again. It just takes anyone with enough distance to have perspective.

But they do need k8 ... on their resume.

I wish we were more willing to offer on-the-job training.

But I've worked with too many people who claim to be Senior or Lead developers but can't actually explain what they do.

I've been tempted a lot lately to try to think of a software team like a sports team. Coach, assistant coach, trainers, and physical therapists all about making you think about your abilities at a different, sometimes philosophical level.

The Surgical Unit idea of Brooks has always bugged the hell out of me. I've known enough nurses to know that you don't want to put surgeons in charge of more than one life at a time and then only for a couple of hours, and letting them interact verbally with those people is a fucking disaster half the time. Not unlike some highly decorated software developers I know. They're brilliant as long as they don't actually have to help people.

The head game in software has been overlooked for far too long and to everyone's detriment. Users as well as producers.

If we had the training part right, this FOMO anxiety would be classed as a disorder.

There is a use case where its really the best solution I've seen so far, say you need to cluster a long running stateful service. Its written in C so making it stateless is absolutely non-trivial. So simply load balancing won't work. Docker swarm could work but compared to Kubernetes stateful sets who is actually using docker swarm or other clustering technology for scaling stateful services.

Kubernetes is WebSphere reinvented then, along with all the management complexity and the expensive consultants?

Great analogy.

Context is always key with these kinds of reactions. Few people feel that hex editors are unnecessary, they probably don't know they exist. The reason you see this sort of thing with Kubernetes is that for whatever reason its hype over-extends its problem domain and many times people who do not need them receive the suggestion (or insistence) to use them. If someone were to tell you to use a hex editor to edit your JavaScript, you might very well reach the conclusion that hex editors are a cult, and useless. Someone might then point out that there are actual completely justified use cases for them. That's what I see happening here: whether it be indirect (tons and tons of blog posts and articles about moving to Kubernetes), or direct (an employee insisting that the company's infrastructure be moved to Kubernetes), or a mix (starting to see Kubernetes experience as a requirement for jobs that probably doesn't need it), all of a sudden you have the backlash against the perceived "Kubernetes for everything" culture (which in turn looks like a weird straw man to people who actually know what its for).

> this isn’t what Kubernetes is really for.

i'm guessing that writing dyson in Nim is a tacit acknowledgement of that: if this were something geared toward production ecosystems, it would be in golang like kubernetes? although there is the helm luafication, so perhaps dyson is part of a fringe of non-golang k8s auxilliaries.

another way to implement this is with a 'static CMS' where there are still static pages except built into a situated deploy. the 'cultish' (cultic? anyway) aspect of k8s appears to be to phrase all the things in terms of k8s constructs rather than using k8s constructs as a foundation and abstracting out.

i learned about 'rollout' from the CI portion of this post, although initial attempts to search for a comprehensible description of it fail.

Honestly dyson is just something I wrote for myself to see how difficult it would be to write. I don't expect anyone else to use it. The tool is also a punny name, because you'd need to terraform a dyson sphere before you(r apps) can live in it.

I just wanted something with easy templating syntax like this: https://github.com/Xe/within-terraform/blob/master/dyson/src...

The fact that cligen (https://github.com/c-blake/cligen) exists too makes it super easy for me to define subcommands of the thing: https://github.com/Xe/within-terraform/blob/master/dyson/src...

Well, maybe it isn't _only_ what it's for, but I have thought of doing largely the same and "bundle" a bunch of assorted projects onto a single 3-node cluster, including my blog.

The only thing that's really prevented me from doing so is that I have my own micro-PaaS (https://github.com/piku) that makes it trivial to run a bunch of different apps/services on the same VPS, and the added complexity isn't really necessary.

But since I deal with k8s practically every day at customers, attrition might be compensated by not having to switch tooling.

YMMV within this sort of scope.

I mean, that’s kind of my point. It’s just not designed for that use case.

Someone could build a simplified fork / derivative of Kubernetes designed for this purpose. That would be pretty rad actually, but it would cease to be Kubernetes because the complexity is the point.

k3s.io is great, and I use that as well - is to be what I will eventually run in "production" for my personal stuff.

k3s? https://k3s.io

> The only thing that's really prevented me from doing so is that I have my own micro-PaaS (https://github.com/piku) that makes it trivial to run a bunch of different apps/services on the same VPS, and the added complexity isn't really necessary.

How does it fare in production? I've got a tiny app with two containers (a frontend an a batch job) - it seems like a decent use case.

My website (https://taoofmac.com) is exactly that (web and batch workers) and has been running on it for almost 3 years now, but bear in mind that it does not use containers - it merely deploys relatively isolated services in virtualenvs (or equivalent).

Since I use CloudFlare (hi jgrahamc!), it's been peachy.

I do not agree with your assessment of Kubernetes. It is not equivalent to something like WildFly or TomEE. Kubernetes runs/manages application servers (and not just java once), along with a whole host of other things devops things at Scale... Kubernetes is great for setting up a blog, and the 11 other applications the author is trying to run.

> complaining that running a blog on Kubernetes is too complex

think you're really missing the point here. Why do you think this was included in the post? https://twitter.com/dexhorthy/status/856639005462417409

Oh I get the author’s point, but her use case was “basically a Heroku replacement for easy deployment”. It’s just the wrong use case for Kubernetes and it is well known that deploying to Kubernetes is a bit of a nightmare.

Except Kubernetes has sucked all the oxygen up in the industry and has had a subset of adherents that trash the alternatives such as Heroku, Cloud Foundry, etc.

As such, people do expect it to replace Heroku.

Anyone who suggests k8s as an alternative to Heroku is wrong. Heroku is a product that manages infrastructure so that the programmer doesn't have to, k8s is a solution for operational engineers that want a code-driven approach to managing their own infrastructure. Suggesting k8s as a replacement for Heroku is like suggesting docker as a replacement for EC2.

And yet, they both fight for the same budget. They're absolutely alternatives in that sense. Not necessarily the right tool for the job, but alas, that's not stopped the wave. And eventually, even Heroku will be based on K8s, as they see the writing on the wall.

Also note that most adherents view K8s as a replacement (more accurately new API) that replaces EC2. the fundamental unit of computing becomes the Pod, not a VM.

What is the right use case for K8s?

And why is "ease of deployment" not something that someone should expect from K8s?

I've been sitting on the K8s sidelines for a bit as things iron out, and I've been deploying it on bare metal on a test bed over the last few days with the intention of using it as IaaS for some of my own apps.

It seems to be what it's meant to do. Keep my app running on infra following the rules I set.

The goal isn't "ease of deployment", the goal is "infrastructure as code" so that application infrastructure can be managed in a way similar to application source code (e.g. PRs, blame, code reviews, CI, rollbacks etc). This helps ops people because it allows them to think about infrastructure as abstract resources rather than as a collection of individual machines with specific designations. With k8s, individual machines become a homogenized resource that do not need specialized provisioning depending on the application they will host.

> What is the right use case for K8s?

It is for making things easy to deploy ONTO the K8s cluster, not for making the cluster itself easy to deploy.

The key is only to use it for cases where the deployment of the containers without K8s is more difficult than the deployment of K8s.

Isn’t the right answer there Knative or Cloud Foundry? (On top of k8s)

I run more than just my blog. I'm also moving discord and IRC bots there too.

Where would you draw the dividing line between where you think it does make sense to use something like Kubernetes and where it doesn't?

There isn’t a dividing line per se; every operating model is going to have different breakpoints. In general though, I would consider Kubernetes an “enterprise technology”. If you’re a startup you’re going to be better off paying AWS/Heroku for one of their more managed services than hiring someone to build / manage a Kubernetes cluster.

A note on this - AWS (or any other host's) managed k8s will not reduce the need to understand what's going on under the hood. It's still k8s, you're just not running the daemons to make it work (which is, arguably, the easy part).

You've signed an > 99.9% availability SLA with a customer and a service like Elastic Beanstalk or Heroku isn't sufficient for your needs.

Technically, the cloud in general probably isn't for you, since none of them guarantee more than 3x9's on any of their services.

Yup, meant to say >= 99.9%. Of course most businesses operate more like what Cloudflare makes explicit with it's "100%" SLA. Architect for 3 nines and then just pay the penalty when the 3 nines architecture doesn't hit 4 nines.

If you need your workload to span over an (possibly growing) number of server hardwares, you may want to use k8s.

There are other options that are good contenders in case of such a workload.

I'm starting to feel that the whether or not you need Kubernetes is closer to old conversations related to whether or not you need a dedicated DBA. Can your organization survive and recover with hourly/nightly backup restores to recover from an incident? Is your replication so complex that you really need a guy to ensure that's never getting into a bad state? Worse case scenario can most people working on the project restore the database to a valid state if something does go wrong? I feel like these questions have similar representatives in whether or not Kubernetes is right for an organization.

Kubernetes also makes more sense if you look at it as a common way for an organization to run applications among disparate teams with a shared operations infrastructure. It provides a standardized model for things to work the same enough to work the same. If you are only delivering one kind/whole organization thing and it all looks and quacks like a duck maybe you should just deliver a duck instead of putting a duck hat on Kubernetes and asking it to quack.

I don't think there is anything wrong with designing an application that would transition easily into Kubernetes but I feel like many of the proposals/PoC I have seen in the last few years are either fresh systems that get consumed by Kubernetes complexity or are poor replacements to systems that already exist and only seem to serve as resume padders for the team architecting the replacement which gets a viking funeral as soon as they leave. Often the latter case is because the underlying architecture and goal of the system is pre-Kubernetes and doesn't fit the model well of having mostly stateless/replaceable pieces.

Kind of implicit in the other responses here are use cases that call for a full cluster and all the complexity that goes along with it.

But running a single node cluster is totally valid. I use one at home to run arbitrary containers (a DNS-over-TLS proxy, VPN server, radius server, network AP / switch manager, etc.). I don't use load balancers and just use host-based persistent storage, which removes the vast majority of the complexity.

I've seen a lot of people get wrapped up in the complexity of wanting to be able to have a persist storage-backed process go down on one node and come back up on another, and that's not unreasonable, but that's a lot of stuff to figure out early on.

If I weren't using this as a single node k8s instance, I might use it to manage VMs, which might be easier to understand but much heavier and more work to maintain. With what I have now, I've got a folder of YAMLs that defines everything to run on my node, and I'm able to easily put all their persistent data in the same top-level dir for easy backup.

I think the perception of k8s might change over time once people realize that it provides a lot of value even if you completely rule out the tougher stuff to do on bare metal (like load balancers and shared persistent volumes).

The best way I can desct K8s is language agnostic J2EE.

Reply on news.ycombinator.com