Nvidia Releases CUDA Python
nvidia.github.io
nvidia.github.io
Here is the pytorch one: https://ngc.nvidia.com/catalog/containers/nvidia:pytorch
They have others for tensorflow etc. I would highly recommend this as the host system only has to have the NVIDIA driver installed with the rest of the dependencies contained in the docker.