Installing PyTorch Geometric Wrt CUDA Version | Hacker News Reader

Installing PyTorch Geometric Wrt CUDA Version

blog.42yeah.is

13 pointsibobev9 comments

I was just going through this today. Surely there has to be a better way of managing python libraries with CUDA kernels than what we currently have.

The worst part is, after spending a couple hours building 5 separate libraries from scratch and making sure all the versions match, there’s no easy way to reproduce your environment on another machine.

Not sure if it helps in your case: for a package worth keeping, I find it useful to build a script out of the commands I used on the machine I first installed it, and then try the script on another machine and improve it as I go along.

> way to reproduce your environment

can't you use docker?

also, from my pile of links:

https://docs.nvidia.com/deeplearning/frameworks/pytorch-rele...

Yeah every 3-6mo we go through a big ecosystem realignment on the Graphistry stack around CUDA, RAPIDS, torch, transformers, pyg, dgl, etc. It'd be a lot harder without docker and mamba!

Besides the rest of the people I work with not using docker making sharing difficult, docker is just another layer to maintain and deal with.

Especially since I don't work on a fixed project for very long and am constantly installing new dependencies, using different data, etc.

Not to mention I've found adding a layer of virtualization can break some already fragile libraries (e.g., I'd be concerned about multi-GPU training over NVLink).

I have found NixOS to be a really straightforward and robust solution for Python and deep learning.

However, for other languages I use, notably Julia, it is not a good choice yet.

I've been meaning to try NixOS personally but I spend most of my time on a shared server, so it's not an option. Thanks for the suggestion though, I'll definitely try it sooner rather than later.

I will give it a try. Thanks for mentioning it.

Can you share more about your workflow? Or point me towards some directions that were helpful to you?

See workflows described in: https://nixos.wiki/wiki/Python.

Reply on news.ycombinator.com