You may want to look at run.house [0] for a pretty powerful solution to many of these problems.

[0] https://github.com/run-house/runhouse