you only need two operators, the kubeoperator, the dataflow operator

and then every task is just a standalone, vertically scalable service on k8s or a giant horizontally scalable compute job