I use NVidia-Docker extensively in my Open Source project Deep Video Analytics [1] when combined with TensorFlow (which allows explicit GPU memory allocation) its unbeatable in running multiple inference models on a single GPU in a reliable manner. Combining this setup with docker volumes on AWS EFS allows simple multi machine deployments.