OP here. Probably some form of container orchestration system like Kubernetes to manage the deployments + resources. There is also specialized ML tooling like Kubeflow to help out. Its still a developing landscape though.

We made BudgetML not to replace the above but to make it easy for scenarios where the need would be to just "get out there and deploy". Do you see value in that too?