> Both Nomad and k8s have ways to do persistent workloads, but they rely heavily on the underlying infrastructure to "move" volumes between throwaway containers

I did this with Nomad by writing a small API to "place" host volumes which Nomad can use for job placement constraints automatically. It works shockingly well for my use-case which I think is somewhat similar to Fly's? A smallish number of containers that require host persistent storage, and don't expect anything to move -- just disappear forever (persistent til' it ain't).