You aren't a real K8s admin until your self-managed cluster crashes hard and you have to spend 3 days trying to recover/rebuild it. Just dealing with the certs once they start expiring is a nightmare.

To avoid chicken-and-egg, your critical services (Drone, Vault, Bind) need to live outside of K8s in something stupid simple, like an ASG or a hot/cold EC2 pair.

I've mostly come to think of K8s as a development tool. It makes it quick and easy for devs to mock up a software architecture and run it anywhere, compared to trying to adopt a single cloud vendor's SaaS tools, and giving devs all the Cloud access needed to control it. Give them access to a semi-locked-down K8s cluster instead and they can build pretty much whatever they need without asking anyone for anything.

For production, it's kind of crap, but usable. It doesn't have any of the operational intelligence you'd want a resilient production system to have, doesn't have real version control, isn't immutable, and makes it very hard to identify and fix problems. A production alternative to K8s should be much more stripped-down, like Fargate, with more useful operational features, and other aspects handled by external projects.