For example we run ceph and use it in kubernetes. We obviously only run replica sets (the kubernetes feature, for those uninitiated) of 1 for these services. There's also some rough edges with locking for dead nodes, but it's definitely good enough for running the likes of Jira or Gitlab have single points of failure pretty much whatever way you scale them (unless you build a HA nfs server first).
Before that, we were running test clusters with just fleet, using custom bash and etcd for service discovery. Works fine, but you're probably better off with frameworks.
Now where things can go horribly wrong is you decide to use a clustered database, run it in kubernetes, thinking it's fine since you can't mount external storage for replica's but you have 3 nodes over 3 zones and you have backups. Kubernetes will quite happily schedule all of your replica's on the same host, meaning a single reboot and you're dead. There are hacks around this such as defining a hostPort, but of course if someone updates the RC you could happily be restoring to backup. You could run two multiple replica controllers to get around this and that'll allow you to mount storage again but that takes a way a lot of the elegance of the framework.
Point is, you can do it, but you'll need a bunch of uglyness and you'll need to be careful. If that doesn't sound great, way until the software properly supports it. :)