I've used uv with pytorch and cuda fine. What problem have you had?

I also use it in docker to build the container.