What do the authors ever mean? How so? The smallest k8s cluster that makes sense has 3 nodes. This means you need to run at least 3 VMs and pay for everything else you need (storage, secrets, egress). And on some cloud providers like AWS, you pay an extra hourly fee just for the privilege of using k8s (that amounts to around $70 last time I checked). So the above statement makes no any sense.
The key thing is that a container needs about 10-20% of the memory of a virtual machine. Generally, a large cluster allows for very efficient "bin packing" so that workloads with different requirements all get distributed so that every VM is loaded to about 80-90%, whereas you'd be lucky to have 10% across your fleet of VMs. (Don't forget to include all the non-prod instances!)
At the tiniest scales, Kubernetes stops making sense because it's too complex to manage.
However, it does scale down to a single node. E.g.: Azure Kubernetes Service (AKS) will cheerfully run with just one Linux box and has no extra hidden costs.
Is there a source for this statement?
Its origins started in Google, which has thousands of developers, and it is roughly speaking a clone of their Borg system, which, again, was designed for that scale.
In my recent experience a ton of tiny startups which should be using Heroku etc are running Kubernetes for no other reason than engineer ego and "that's what the big boys are doing".
And you say Heroku, "etc", but the thing is that there is very little heroku-like alternative. What "etc" is there really?
The real alternative is a Linux VM with docker-compose.
Once you outgrow that, managed kubernetes is the logical next step, even for a small team.
Kubernetes is a surprisingly reasonable option for an organization with 5-10 developers that has outgrown Heroku-like services.
I don't think I've ever budgeted more than $150/month for a managed Kubernetes controller that Just Works, and that was a relatively expensive one. Everything was preconfigured and we could get started in 20 minutes. The actual servers in the cluster were just ordinary cloud VMs with locked-down images, and they were managed automatically.
Actually running an app on Kubernetes requires two things: (1) a Dockerfile, and (2) a YAML file listing the Kubernetes resources for the app. The YAML file might be anywhere from half a page to 2 pages for a simple one. There's an O'Reilly book that explains what to put in them.
I've also deployed large applications using Chef and Terraform. These often require far more configuration than slapping something on Kubernetes, and Chef offers many more ways for things to go wrong.
However, I do have several rules for Kubernetes happiness:
1. If your app is simple enough to run on a Heroku-like service, do that instead.
2. If you do choose Kubernetes, buy and skim the O'Reilly book, or a similar book.
3. Pay a cloud vendor for managed Kubernetes.
4. Do not mess with the networking config. Almost every Kubernetes horror story I've heard involves network overlays.
I'm not saying anyone ought to use Kubernetes, but it's surprisingly reasonable and pleasant for smallish full-time teams if you follow the rules above.
Managing k8s at the smallest scales is actually easier IMO. If you're using a managed k8s (you should be) then there's not much to actually manage. I've been using single-node clusters to great benefit for my business for years. Let's me focus all my time on billable hours and product updates. Huge force multiplier for me as a solopreneur.
For small scale hosting who do you recommend? I have a few projects I want to get going and thinking it might help to be able to leverage my existing kubernetes knowledge. I always figured it would be too expensive to go with kubernetes for a small project.
The utilization numbers only show that a particular administrator doesn't know how to use technology X, but knows better how to use technology Y or that the defaults in technology Y happen to work better for a particular benchmark (and the administrator doesn't know how to use either X or Y).
I believe that the later is usually the case. Compounded by laziness and lack of real financial incentive to optimize every last bit of the existing infrastructure.
The situation will, however, flip on its head if you want to build a reliable high-performance system. In this case Kubernetes will be an impendiment. Nothing it does it does well or can be optimized / configured to do well. When push comes to shove, you'll have to either get rid of Kubernetes, or lose the race.
K8s basically gets cheaper the more you run (as cloud costs, setup costs, engineering, bin-packing, volume discounts stack up), whereas Heroku basically scales linearly.
Something like k3s makes a lot of sense for development environments though. It is much easier to create a production like environment with something like k3s than it ever was using VMs.
I do feel like there's room in the middle for something more flexible than Ansible and Puppet, but less complex than Kubernetes. If you're just running a few VMs, "upgrading" to Kubernetes doesn't seem like it's worth the hassle. On the other hand, deploying the same application, or applications to say 4 or 8 VMs, that starting to get i little annoying. I'd argue for a tool that augments Docker-Compose, one that understands distribution of applications, but doesn't deal with networking (the network team can handle that). I suppose a little like Nomad?
As a solopreneur, I've saved so much time and money letting k8s run my workloads while I rack up billable hours and build products.
Frankly it is not painless at all. Ingress is complicated and there is a ton of way to configure it, with all the forced tie in with the cloud provider's offering, be it l3 or l4 load balancing, various proxy services, etc.
It's not easy at all either on docker-compose. You have to roll your own proxy container and take care of TLS termination, cert renewals, manage specific headers and config, etc.
I am not sure what exactly I want to add to the conversation, only that either way, it's not "easy". I'd be glad to be proven wrong if you have a specific solution that makes ingress really painless.