The best way to setup tensorflow for gpu is to use nvidia-docker and pre-built docker images. They mention it in their documentation.