I can't imagine a worse combination than Kubernetes and stateful connections.
But the author only uses it to keep alive a couple of IRC connections (which don't send you history or anything on re-connects) and to automatically backup their "huge" chat logs (seriously, 5 GiB is not huge, and if it's text then it can be compressed down to about 2 GiB — unless it's already compressed?).
That's not really the problem — if, say, one of your nodes drops dead (or just drops off the network), the clients' connections also drop, and they all try to reconnect. That just happens and there is not much you can do to prepare for it except by having some idle capacity already available.
Unless you're talking about rolling out strategies for deployment updates, and to be fair I don't remember the controls for that being all that useful but that was 2 years ago, so perhaps things are better now.
If you have a static number of nodes and don't have to worry too much about things autoscaling, I don't see why it couldn't be really stable?
Check out how services, load balancers and the majority of CNI actually work then.
Kubernetes was designed for stateless connections and it shows in many places.
If you want it to do stateful connections you could use something like Agones which intentionally bypasses a huge amount of kubernetes to use it only as a scheduler essentially.
Applications that act "native" and don't need a lot of the extras...
Well, they arguably mostly use just the scheduler then :D
Maybe I’m doing it wrong?
But you do not need a "complex" CNI. Originally k8s pretty much worked with assumption you can route few subnets in good old static way to the cluster and that's it, and it works with that kind of approach still - each node gets a /24, there's a separate shared /24 (or more) for services, etc.
The complexities came from the fact that a lot of places that wanted to deploy kubernetes couldn't provide such a simple network infrastructure to hosts, then later what was a workaround got equipped with various extra bells&whistles
Second: they use a version of node ports that bypasses CNI, so you directly connect to the process living on the node. This means there’s no hiccups with CNI if another node (or pod) gets unscheduled that had nothing to do with your process.
In most cases, web services will be fine with the kinds of hiccups I’m talking about (even websockets); however UDP streams will definitely lose data - and raw TCP ones may fail depending on the implementation.
No, why do yours? :D
If you're using cluster autoscaling with very small (or perfectly sized) nodes, I could see it being more of an issue on a busy cluster.
But even then, I wouldn't set up a database to auto-scale. A new node could get created, but it doesn't mean the db pods will be moved to it. They'd ideally stay in the same location. And on a really busy cluster, I'd prefer a separate node pool for stateful apps.
Using something like Stackgres makes it relatively painless to run postgres in k8s too, it handles setting up replicas and can do automatic failover.