Hacker News Reader

K8s doesn't help you solve your geographical region problem, because the geographical region problem is not running appserver instances in multiple regions. Almost any PaaS will do that for you out of the box, with way less fuss than k8s. The hard part is distributing your data.

Less overhead than writing your own glue code, less overhead than learning Kubernetes, is just use a PaaS like Google App Engine, Amazon Elastic Beanstalk, Digital Ocean App Platform, or Heroku. You have access to the same distributed databases you would with k8s.

Cloud Run is PaaS for people that like Docker. If you don't even want to climb that learning curve, try one of the others.

PaaS get such a bad rap from devs in my experience, even though they would solve so many problems. They'd rather keep their k8s clusters scaled to max traffic and spend their nights dealing with odd networking and configuration issues than just throw their app on Cloud Run and call it a day.

> just use a PaaS like Google App Engine, Amazon Elastic Beanstalk, Digital Ocean App Platform, or Heroku.

This is the right way for web most of the time, but most places will choose k8s anyway. It’s perplexing until you come to terms with the dirty secret of resume driven development, which is that it’s not just junior engs but lots of seniors too and some management that’s all conspiring to basically defraud business owners. I think the unspoken agreement is that Hard work sucks, but easy work that helps you learn no transferable skills might be worse. The way you evaluate this tradeoff predictably depends how close you are to retirement age. Still, since engineers are often disrespected/discarded by business owners and have no job security, oaths of office, professional guilds, or fiduciary responsibility.. it’s no wonder things are pretty mercenary out there.

Pipelines are as important as web these days but of course there are many options for pipelines as a service also.

K8s is the obviously correct choice for teams that really must build new kinds of platforms that have many diverse kinds of components, or have lots of components with unique requirements for coupling (like say “scale this thing based on that other thing”, but where you’d have real perf penalties for leaving the k8s ecosystem to parse events or whatever).

The often mentioned concern about platform lock in is going to happen to you no matter what, and switching clouds completely rarely happens anyway. If you do switch, it will be hard and time consuming no matter what.

To be fair, k8s also enables brand new architectural possibilities that may or may not be beautiful. But it’s engineering, not art, and beautiful is not the same as cheap, easy, maintainable, etc.

This has got to be the most out there k8s take I've read in a while. k8s doesn't save you from learning your cloud providers infrastructure, you have to learn k8s in addition to your cloud provider's infrastructure. It's all ALBs, ASGs, Security Groups, EBS Volumbes and IAM policy underneath and k8s, while very clever, isn't so clever as to abstract much of any of it away from you. On EKS you get to enjoy more odd limitations with your nodes than EC2 would give you on its own.

You're already building on a cluster, your cloud provider's hypervisor. They'll literally build virtual compute of any size and shape for you on demand out of heterogeneous hardware and the security guarantees are much stronger than colocated containers on k8s nodes.

There are quite a few steps between single server and k8s.

You can self host Kubernetes on "dumb" VMs from Hetzner or OVH.

K8s was designed around deployment on premise on bare metal hardware.

The cloud extensions were always just a convenience.

The same argument can be made about Borg. Someone at Google needs to know about things like Juniper switches and hard drive bays. Someone needs to repair or replace defective compute nodes and power switches. Before they had software load balancing, someone had to manage the hardware load balancers.

But the idea of Borg is that all of that's abstracted away for the typical developer. It's the same with k8s. The infrastructure team in your org needs to understand the implementation details, but really only a few.

You can also configure load balancers, IAM groups & policy etc as k8s CRDs. So all that stuff can be in one place in the code base alongside the rest of your infrastructure. So in that sense it does abstract those concepts. You still need to know something about them, but you don't have to configure them programmatically yourself since k8s will do that.

Yep, it's a cluster OS. If you need to run a cluster, you need to explore and understand the trade offs of k8s versus other approaches. Personally I run a small cluster on k3s, for internal tools, and love it. Yes it's a load of new abstractions to learn, but once learnt if really helps in designing large scalable systems. I manage lots of pet machines and VMs for clients, and it would be soooo much easier on k8.

I like to describe it similarly, but as a way of building platforms.

Exactly. I've come to describe k8s as a distributed operating system for servers.

K8s tries to abstract away individual "servers" and gives you an API to interact with all the compute/storage in the cluster.

What is the container orchestration tool of choice beyond docker swarm, then?

Is nomad still around?

Thanks, hadn't heard of that.

Seems pretty active per its commit activity: https://github.com/hashicorp/nomad/graphs/commit-activity

But the fact that I hadn't heard of it before makes it sound not very popular, at least not for the bubble I live in :).

Does anyone have any practical experiences to share about it?

Yes have a few Nomad clusters in production and it's been great.

You'll certainly want to combine it with Consul and use Consul templates and service discovery though.

I'd say the difficulty and complexity level is between Kubernetes and Docker Swarm, not having to use YML too is a big benefit imho.

If that's practical experience that you need, I remember reading some studies comparing the reliability and efficiency of nomad vs k8s under load (spoiler: they do not look very good for k8s).

If that's popularity that you need, then sure, nobody ever got fired for choosing kubernetes.

> Containers are an implementation detail.

They really aren't.

Personally I have a big Nix derivation to deploy my (heterogeneous) cluster to bare metal.

None of the k8s concepts or ideas apply here.

how does it work? how many nodes? what do you use for consensus, request routing, etc...?