I think if you run a cluster in a trusted environment it should be more efficient to use ray or something similar