NVidia has special support for making their drivers work in Docker containers:

https://docs.nvidia.com/datacenter/cloud-native/container-to...