It’s Time to Embrace Kubernetes Really?
pulumi.com
pulumi.com
My friend and colleague, Paul Stack: https://twitter.com/stack72 are at very opposite ends of the spectrum when it comes to Kubernetes. We put our heads together and tried to create a balanced take on Kubernetes where our opinions didn't cloud the article. It's obviously very polarizing, but I think we managed to ask some good questions before you started the adoption process.
Would love to hear more via twitter too! https://twitter.com/briggsl
I'm not sure I agree about templating YAML files–why is templating JSON any better, really?–but YMMV.
people think they're being smart by telling you to check whether such and such fits your needs. usually it's not bad advice. but the deck is pretty stacked here. nothing else has shown itself to be as flexible, as extensible, as consistent. well demonstrated but it's capability at handling many different add-ons (controllers, operators). rarely is cloud architecture do readily deployable to other non-core concerns, especially via the core tools & constructs.
if you want to go march off in to tech that does less, that will only do a little bit, that less people will be familiar with, that has less community, sure. spend a couple hours justifying to yourself why you are not going to use the really amazing all encompassing tech that's quickly supplanted everything else. pick some other combination of technologies which you will assemble, make your own stack, yeah, you're an independent thinker, you don't need convention or all this extra stuff! it's just gonna slow you down. go for it.
Kubernetes is not easy. but I find arguing against it to be difficult. there have been tools upon tools upon tools for so long. but little out there has brought the different concerns together, found overarching paradigms to create a basic, trusted quality of life that Kubernetes has clarified. again, it's not easy, but learning it ought to keep paying forward again & again, as you pick up new interests, want new services. even if those are Amazon services, ACK now will let you build Kubernetes deployment units that include those AWS services. the paradigm works.
It feels awesome from a developer standpoint once you work out all the kinks. It's very natural describing your service with a bit of yaml, and then being able to then spawn it into your infrastructure and have it automatically scale out.
If there was a cost effective k8s cluster you could rent then that would be the thing that finally kills traditional infrastructure for me.
A car is a hugely complicated machine and I'm sure in the 1910s people said only some people will ever need them. Bad analogy considering cars kill lots of people but take that part out :)
After your single cattle is murdered, the next one has to be raised right?
State has to come from somewhere. Downtime has a cost for most businesses.
But being able to blow away a k8s node and bring it back up in 2 minutes (or have k8s auto-spawn a replacement node) is WAY faster than having to reinstall wordpress, reinstall nginx, set IP addresses, fiddle with your DNS, restore a database from a backup, etc, etc, etc.
Kubernetes just makes that story so much simpler. Heck a good cloud provider can just handle any machines fucking up automatically without any human intervention on your side.
The same can't be said for a single machine.
Managing a cluster is the tricky part of k8s but just running your stuff on an existing cluster is extremely straightforward. I could definitely see the attraction of a cheap managed k8s cluster with a single node. Maybe hosted k3s or something.
https://www.digitalocean.com/products/kubernetes/
https://www.scaleway.com/en/kubernetes-kapsule/
IaaS model is more 'everything is chargeable in small units', a free control plane doesn't really gel with that IMO, and also they have some huge users that it'd cost a lot to run the control plane for, pushing up the price of what you connect to it (or something must give) and making more PaaS-style like DO & Scaleway anyway.
Make sure to use metal LB or all your ingresses will be served with Google LBs that are hecka expensive.
I forgot to tear down a small EKS cluster last month. 4 days, 60€.
I run a 3-node Kubernetes cluster on Google cloud. It costs me around $100 per month (with 1-year reserved instances). Not terribly expensive.
Openshift is big in on-prem enterprise setups because as flawed as the install process is or as ridiculous licensing is, you get a remarkably complete system pretty easily out of a bunch of servers that integrates with things like your corporate LDAP.
I think part of the issue is that many of the bigger players want you to use their managed k8s solution and not just give you their setup - I think that's a temporary thing. A few of the big boons of k8s is the ability to be multi-cloud, and vendors are losing out on a revenue stream by not embracing that (though, AWS has a new multi-cloud k8s thing, so maybe they get it).
Some of our teams have been drifting naturally towards ECS Fargate and found it to be a great fit. Almost invariably the k8s question comes up, and it's usually a reaction to the popularity of k8s and EKS. It's quite easy to start doubting your own well thought-out decisions in the face of a large popular elephant sitting in your rooms which wants to trample on everything.
ECS is definitely more suited for small use cases however.
One can learn the developer side of K8's in a week or so enough to be deploying apps and setting up services and pods without the overhead of managing etcd and distributed clusters.
Setup = rep groups, internal services, etc? You need k8s-savvy ops people, or really good devs who like ops stuff.
They are called DevOps Engineers and it's a standard role these days.
And it doesn't matter whether they like it or not. In many organisations these days developers own their piece of work all the way through to production and are expected to provide an ops-like support for it.
In an ideal world, you could purchase or rent commoditized resources (compute, storage, bandwidth) and have those resources deploy your application in a standardized way. K8s is so close to that out of the gate, that having to use cloud provider's expensive and proprietary offerings feels sub-optimal.
I agree that cloud offerings are the way to go for anyone without good reasons not to (eg compliance, lack of connectivity, or full control over upgrades). But a lot of the "I prefer on prem" attitude comes from realizing that having a black box of an ingress controller on my cluster isn't the end state of how apps get deployed
But people still use the cloud.
OTOH if a service is available and would work for someone's use case, by all means, use the service. I've set up native k8s clusters just as exercises, and it's no picnic–and I didn't need to manage those clusters, later. Just doing build exercises alone was enough to scare me straight.
* Easy simulation of networks between cgroups.
* A mechanism for splitting a ulimit across kernels.
* Hot swappable kernels and kernel tuning parameters so machines can be quickly re-configured.
https://thenewstack.io/unfixable-kubernetes-security-hole-me...
You can't trust the cloud provider:
https://unit42.paloaltonetworks.com/windows-server-container...
https://media-exp1.licdn.com/dms/image/C5622AQH70okwLkMlNg/f...
What about the concept of not having roommates? Plenty of infrastructure alternatives exist besides "containers".
So in your world you have to only use bare metal ? Sounds fun.
EC2 basic instances are multitenant, but there are such a thing as EC2 dedicated instances (and dedicated hosts), which run on single-customer reserved hardware.
It's a bug but a very minor one. Especially if you are securing traffic inside the cluster.