Kubernetes 1.11 includes In-Cluster Service Load Balancing and CoreDNS

I actually switched all our servers from hetzner to aws because for our needs it was cheaper and I could spend my time on things I care about. For me K8s only handles non persisting data. All data other is stored outside (Managed DB (like RDS) or S3) of it. You only need to backup the masters. If you use something like GCE that is managed for you & free.

Each node you are going to include in your cluster needs to have docker and kubelet running. You will need to start kubelet with your init system (systemd, etc).

Then all you need is to bootstrap a control plane. The control plane has 3 components. The API server needs to communicate to etcd, everything else just talks to the API server. The API server is stateless and can (should) have multiple instances running. The other two parts are the Controller and Scheduler. Also stateless. The tool Bootkube[1] can generate all the configuration and perform the bootstrapping for you.

To answer your dot points, all but 4 are easy enough with using the kubernetes docs and some systems/networking knowledge. Point 4, assuming you mean starting new machines based on load, will require you to look at which systems are supported by the autoscaling system and potentially add an integration for your environment. There is the horizontal pod autoscaler that will run more of your service when needed and the cluster autoscaler[2] which will start more machines when it can't run as more instances of your service on the current number of machines.

Edit: There are also tools like Patroni, Stolon, and postgres operators that will assist with your DB (postgres in this case) management. Scaling, HA, backups, etc.

1 - https://github.com/kubernetes-incubator/bootkube

2 - https://github.com/kubernetes/autoscaler/blob/master/cluster...

Of course. There are literally dozens of distributions and repackaged versions now[0]. The advantage of GKE et al is that they keep them up to date -- still a bit harder than it looks.

> Be cheap

It very much depends on what "cheap" means and what your needs are. GKE with a cluster autoscaler is fairly inexpensive for what you get. But if your time is free and you like to tinker, self-managing on a VPS might be for you.

Be prepared to invest a fair amount of time. Kubernetes is simple in each part, but there are lots of parts to learn.

> Automatic failover (This should cover majority of downtime imho)

This is more of an app concern. Kubernetes itself won't really do this for you. Something like Istio will probably also be necessary to help with traffic management.

> Backups

Very much up to the distribution / hosting environment.

> Automatic scalability (CPU over 95% for last 2 minutes or smth like that? add new containers pls)

Horizontal Pod Autoscaler; a Vertical Pod Autoscaler is under development. Other autoscalers are being developed for more specialised uses (I've worked on one) and I expect there will be some concept of pluggable autoscaling soon.

> Load balancing

Numerous solutions in this area. Typically left to the IaaS. Istio can do this.

> How are K8S backups handled? ... several containers spawned of the DB

Regarding DB backups: I would again use a hosted service if at all possible. Several containers can share a volume in a Pod or across Pods. But long-running stateful services is not yet quite the Kubernetes sweet spot.

> What restarts K8S orchestrator if it fails?

Again, numerous solutions. I work for Pivotal, our solution in PKS is to run everything with BOSH. Something like GKE uses Google's own monitoring and recovery systems.

I should add here that I don't think GKE is necessarily the "best" of the public hosted options. Just the only one I've used in anger so far. I actually prefer to do development work on minikube whenever I can. If you go down that route, shop around!

[0] https://kubernetes.io/partners/

There are cheaper baseload alternatives appearing, but it'll probably be 12 months until they're stable.

https://labs.ovh.com/kubernetes-k8s

https://github.com/hetznercloud/hcloud-cloud-controller-mana...

It handles HA-setup, etcd member replacements, hardened configuration for enhanced security (follows NIST SP800-190 recommendations). You can easily extend the installation with addons of your own or from any Kubernetes ecosystem project

(Disclaimer: I'm one of the contributors)

Your bare metal hosts can be provisioned as Rancher hosts, which are dumb slaves managed by a Rancher server.

From the Rancher server, you can trivially express "I want this Docker container deployed across n Rancher hosts" or "I want 1 instance of this docker container across all Rancher hosts with the tag "has public IP".

You can group your Rancher hosts into Sandbox and Prod Rancher environments. You can easily install Rancher's load balancer service on them, or mount network storage, or register secrets upon them (like private Docker registry keys).

It also gives you health checks, host monitoring, and zero downtime redeploys. Super easy to use from the UI or CLI. Easy to install, too.

The two big things you get on the cloud that you need plugins for on bare metal are networking and storage. Networking plugins are basically fire&forget these days. Storage is definitely not at that stage yet. You can use a plugin that provides redundancy & failover like Rook/cephfs or Heketi/gluster or you can use local storage and handle redundancy/failover similar to how you'd do it without Kubernetes.

- Kubernetes is like any other abstraction. It complicates things and adds another layer to manage and understand, but it lets you manage disparate server apps similarly. So it really only makes sense if you have a good number of disparate server apps to manage.

- If you're looking for 5 nines or something like that, as a small operator you're more likely to screw things up through a PBKAC or misconfiguration than you are via a hardware failure. So you probably want two clusters with manual failover between them. That way you can switch to one cluster while you operate on the other. The second cluster could be a GKE cluster scaled down, but that's still a bunch of complexity and some expense. You're talking about zonal failures, so you'd need two clusters to protect against that anyways, so...

- you probably need several layers of backups. An etcd backup, ark, and your persistent storage backups. Ark handles persistent storage backups on the cloud, but if you're on bare metal, you need to do that yourself the old fashioned way.

https://rancher.com/what-is-rancher/what-rancher-adds-to-kub...

DigitalOcean is also working on a managed kubernetes service, which should be less expensive than AWS,GCP,etc

I agree that the container clustering world tends to focus on stateless scenarios, conveniently ignoring the "hard stuff". However, that doesn't mean it can't be done. k8s can be told to maintain persistent volumes that can be claimed by containers. This mechanism can be (has been) used to implement "real" DB HA. e.g. see this talk : https://www.youtube.com/watch?v=Zn1vd7sQ_bc

I do it with kubeadm and metallb (for my public ip) I'm not sure how good ip failover will work on hetzner/ovh, but on your own hardware in your own network it is not a problem.

(K8s does not need backups, only your etcd and configs should be backupped but it is unlikely that you loose your whole control plane in one step.)

3)IMHO even for small setups because you buy into a <pseudo standard> ecosystem. Backups can be handled for example by stash[0] instead of having to (partially) write your own backup solution.

4)I haven't tried this on OVH, but as it's a openstack based environment certain items are well integrated e.g. persistent volume handling [1]

[0]https://github.com/appscode/stash [1]https://kubernetes.io/docs/concepts/storage/storage-classes/...

To answer your question briefly: Kubernetes is hard to stay up-to-date, but it aims to improve later on. Kubernetes was largely created to solve problems scaling physical hardware -- I would not setup Kubernetes unless you run into a problem where you are growing out of dedicated host offerings thus need to update your apps across multiple such devices (easier options exist, look at LXC clustering). You need to define "reliable". If it is just your app or hardware crashing, you can probably get away with two dedicated computers in different colocations.

My wife's company website ran on it for years.

Or you can have a startup with a small team and small number of users, but the legal and market requirements may dictate things like availability requirements for which k8s can add a lot of value that would normally cost you hiring multiple ops folks and writing a lot of boilerplate code.

For the latter kind of business, k8s is probably a good investment that is worth the added costs of running on a more expensive cloud provider. You’ll probably have to stick to them for security/compliance reasons anyway.

For the former type of company... k8s is probably only worth chasing if you are already seeing scale, or if you see becoming more like the latter. In that case, I would first test my app on something like GKE to see what it means from that perspective. Then I would look at deploying on my preferred hosting provider.

If you can’t get your app to run well/how you like on GKE, it’s probably not worth planning a cluster deployment just to find out the same.