Yeah, I recently spied the 4 node Jetson Mate carrier board on CNX that facilitates building your own Multi GPU cluster ;)
https://www.seeedstudio.com/Jetson-Mate-Cooling-Kit-p-4784.h...
They recommend Kubernetes for private cloud workloads. But I am wondering if there isn't room in the ecosystem for a new virtualization / containerization technology to orchestrate tasks on multi gpus? Have heard NVidia is developing its own OS, or rather two, one for Cloud AI and one for real time robotics, but its still research phase...