Hacker News Reader

Comment by Thaxll | Hacker News Reader

> Have you ever tried setting up a Kubernetes cluster and deploying apps in it? Kubernetes doesn't save work

This is wrong, deploying on Kubernetes is easy and quick for most apps, you have one docker image one deployment spec and that's it.

https://kubernetes.io/docs/concepts/workloads/controllers/de...

View on HN

And a service configuration, ingress, other networking, persistent volumes, a mechanism for updating deployed applications, management of the nodes (even with a managed service like EKS or other cloud solutions), logging, roles, security, etc. If you're a developer and all you have to worry about is one deployment spec, thank your devops team for making your life easier. Kubernetes is great for making the dev team's life easier, but someone did a lot of work to make it that easy for you.

Couldn't agree with this more. At my last company there was a fair bit of time put into making the deployment process for our microservices as simple as added a new YAML file to the deployment repo. That file pulled a custom chart, and as a dev you just needed to configure a few variables to get everything up an running. But if you were deploying something that couldn't use one of the pre-configured charts it was a bit more work, especially if you've never done it be hand before.

Probably 98% of the devs were blissfully unaware of that complexity that the charts abstracted, and it let them focus on the services they were writing. I wasn't one of them, and always made sure to thank the devops team for simplifying the day to day deployments whenever I had to deal with writing a custom one.

You can now do something similar with Bunnyshell.com

It Handles the devops jobs for dev teams.

Full disclosure, I work for Bunnyshell.

In the absence of any description of what “something similar” or “handles the devops jobs” actually means, this comes across as spam, not informative.

Sorry about that, I should have been more informative.

Bunnyshell makes it easy to create and manage environments. (EaaS - environments as a service)

You connect your k8s cluster(s) and git accounts/repos, it reads the docker-compose files and creates deployments on the cluster.

You don’t need to know or write Kubernetes manifests, those are created for you.

You also get auto updates and ephemeral/preview environments (when a PR is created against the branch of your env, Bunnyshell deploys a new env with the proposed changes).

You are not restricted to creating resources only on the cluster, you can use Terraform for any resource that is external to the cluster ( like S3 buckets, RDS instances, anything Terraform can handle).

Hope this helps,

THIS. In my homelab, I spent roughly a day cutting services over from Docker-Compose to Kubernetes. That day included writing the Helm templates for everything, bootstrapping three bare metal nodes with a hypervisor (Proxmox), clustering said hypervisor for HA on its own isolated network, making images and then installing K3OS onto six VMs across the three nodes (3+3 control plane/worker), installing and configuring persistent distributed storage (Longhorn) with backups to a ZFS pool as an NFS target (Debian VM configured via Packer + Ansible), configuring MetalLB, and smoke testing everything.

A day's work for one person to accomplish that isn't bad, IMO, but what that doesn't capture is the literal weeks I spent poring over documentation, trying things, running tests, learning what didn't work (Rook + Ceph is a nightmare), and so on. I went so far the day before the cutover as to recreate my homelab in Digital Ocean and run through the entire installation process.

Having services that magically work is hard. Having a golden path so you can create a new one with a few clicks is even harder.

This is still easier with Kubernetes than with other tools. Installing e.g. fluentd and Kibana is again a configuration / chart which you apply to the cluster. For monitoring and visualization again you have an operator / chart that you can apply to the cluster. Yes there is a lot of complexity and learning involved but overall I was still pretty amazed at how quickly I was able to get a cluster up to speed with the available tools.

Still need to maintain the cluster though, and boy does it require some fun maintenance. Kubernetes is more than just a distributed docker image scheduler, you also need to install and maintain basic dependencies like the pod networking implementation (a whole virtual IP space!!), DNS, loadbalancing, persistent volumes, monitoring, etc etc. Maybe your cloud provider sets it all up for you, but you're not going to escape having to debug and fix issues with that stuff, whether they be show stopping bugs or performance-impacting inconveniences like higher latency.

  > the pod networking implementation (a whole virtual IP space!!)

That part, at least, can be made simple: https://john-millikin.com/stateless-kubernetes-overlay-netwo...

  > DNS, loadbalancing, persistent volumes, monitoring, etc etc

None of that is part of Kubernetes, and you'll need it (or not) regardless of how you choose to handle process scheduling.

There's a sort of common idea that a "Kubernetes cluster" is an entire self-contained PaaS, and that (for example) monitoring in Kubernetes is somehow fundamentally different from what came before. It's easy to fall into the trap of creating an internal clone of Heroku, but Kubernetes itself doesn't require you to do so and it can be a lot faster to just run Nagios (etc).

> None of that is part of Kubernetes

Well, load balancing is part of Kubernetes (except that it does not provide a sane implementation), and tbh having to debug strange failures caused by seemingly-innocent Service configuration is my least-favorite part of Kubernetes.

I agree with you on other points.

DNS too

You'd weep if you see an actual deployment yaml. What documentation provides is something minimal, and that's very long for something minimal.

Also, self-healing can create interesting problems which are fun to trace, debug and understand.

"Before you begin, make sure your Kubernetes cluster is up and running."

"And if it's not, please roll-back your systems and install it from scratch."

I'm expecting you are getting downvoted because of your assumptive and dismissive statement. The type of deployment you are suggesting is overkill to apply k8s as there is minimal to no orchestration.

Also because they are just outright missing the point of the post they were replying to. Yes, deploying thing on kubernetes is pretty simple. Deploying kubernetes itself is definitely not very simple.

People are confused, you should most likely not opperate k8s yourself if you don't know or does not have a team for it, nowdays managed k8s are easy to use and operate, with 3 clicks you get a cluster up and running with sane defaults.

Not to sound trite but I feel like your continued pressing of this issue evokes the entire "Do you even lift, bro?" meme. There are PaaS that can get you up in 3 clicks, but now you are in bed with k8s which is critical for the infrastructure. I think it's quite necessary to definitely understand the inside-and-out of such a vital component for my infra.

Either you’re so good with k8s and are clueless about how rare that expertise is, or more likely, you’re so clueless of how complex and fickle k8s can be if configured wrong but you’re not even aware that you don’t know.

I’ve seen actually decent engineers (maybe they’re not decent?) bring down prod because they accidentally Kubectl deploy’ed from their command line.

Reply on news.ycombinator.com