I guess my main question is would anybody actually use preemptive for k8s knowing that if a worker node gets reaped k8s will magically route only to healthy nodes while the preempted instance replacement is standing up?
First of all, GK3 would create new nodes when one of them gets killed.
Second, builidng “cloud-native” applications means to build stateless applications, e.g. using 12-factor. So it does not really matter if your application gets killed, restarted or horizontally scaled.
For databases, i always recommend to use a PaaS offering to have an easy and worry-free life
Just make sure your app has proper probes, and make sure the probes returns 500 when the process receives a sigterm, and it should be invisible to you.
Server-less will be remembered as the most embarrassing thing humans have ever invented.