Big mistake. Overnight, the cluster config files I used were no longer supported by the k8s version DigitalOcean auto upgraded my cluster to and _boom_. Every single business was offline.
Made the switch to some simple bash scripts for bootstrapping/monitoring/scaling and systemd for starting/restarting apps (nodejs). I'll never look back.
My occasional moral dilemma is idle power usage of overprovisioned resources, but we've found some interesting things to throw at idle hardware to ease our conscience about it.
1. Shovel salesman insisting all "real" gold miners use their shovels
2. Those that have already acquired shovels not wanting their purchase to be mocked/have been made in vain.
Neither are grounded in reality. Why people believe their tiny applications require the same tech that Google invented to help manage their (massive) scale is beyond me.
Which was it?
And ultimately, it wasn't hard to upgrade, even if you deal with auto-upgrading cluster and forgot about it, because "live" deployments got auto-upgraded - you do need to update your deployment script/whatever though.
I had a heck of a time finding accurate docs on the correct apiVersion to use for things like my ingress and service files (they had a nasty habit of doing beta versions and changing config patterns w/ little backwards compatibility). This was a few years back when your options were a lot of Googling, SO, etc, so the info I found was mixed/spotty.
As a solo founder, I found what worked at the time and assumed (foolishly, in retrospect) that it would just continue to work as my needs were modest.
Now, I just build my app to an encrypted tarball, upload it to a secure bucket, and then create a short-lived signed URL for instances to curl the code from. From there, I just install deps on the machine and start up the app with systemd.
IMO, Docker is overkill for 99% of projects, perhaps all. One of those great ideas, poorly executed (and considering the complexity, I understand why).
Damn, that's the dream right there
It's only good for very large scale stuff. And then a lot of the time that is usually well over provisioned and could be done considerably cheaper using almost any other methodology.
The only good part of Kubernetes I have found in the last 4 years of running it in production is that you can deploy any old limping crap to it and it does its best to keep it alive which means you can spend more time writing YAML and upgrading it every 2 minutes.
A pair of load-balanced web servers and a managed database, with Cloudflare out front, will get you really, really far.
Sounds like user error.