I am the lead devel of pytorch. No you dont need to install CUDA to get pytorch to work on GPUs, we ship the relevant bits with our binaries. All you need is a working NVIDIA driver.