Rewrite all of your jobs using Apache Beam. Then use whatever runner you want: Spark, Flink, Google Cloud Dataflow, etc.