Hacker News Reader

Comment by Foobar8568 | Hacker News Reader

>Please use CUDA 11.4

Which should be compatible with any Kepler late architecture, as in 6xx models from 10years ago+?

The real issue is having to keep multiple copies of CUDA on a system to satisfy all the different moving parts that each want different CUDA versions and CUDNN versions, and those different CUDA installers fight over and uninstall each others' nvidia driver versions because CUDA is shipped with the nvidia driver, bundled and listed as an apt dependency.

My system changes from nvidia-525 to nvidia-535 to nvidia-520 to nvidia-515 on a daily basis because I need to reinstall a different CUDA version just to try some new paper's code.

PyTorch did it right, it now ships with its own CUDA and doesn't take a shit about version what you have in /usr/local. Everything else should do the same.

Don’t install the drivers with CUDA? Don’t rely on distro-packaged CUDA?

NVIDIA's official CUDA is packaged with their damn driver.

Hi, you can choose not to install the driver with a CUDA install, and to download the driver separately.

Only if you use the runfile, but that breaks yet other packages that specify 'cuda' as an apt dependency.

Your 'cuda' packages should use '>=' not '=='

e.g. 'cuda-12' should depend on nvidia>=525 NOT nvidia==525

That's a fair point. I'll look into it.

Hi, I looked into this and it seems that it is indeed using >=

Depends: cuda-libraries-11-8 (>= 11.8.0), cuda-drivers (>= 520.61.05)

Hm no, what you actually have is cuda-libraries-11-8 depends on cuda-drivers and cuda-drivers depends on cuda-drivers-520 = 520.61.05-1 where you have the version number repeated in the name. That is a really bad pattern for things intended to be backward compatible, as multiple installs of cuda will overwrite "cuda-drivers" and each is pegged to a certain "cuda-drivers-X" where X is part of the name. This results in some really weird dependency hells if you try to install more than 1 CUDA version at the same time depending on the order you install them.

    $ sudo apt show cuda-drivers
    Package: cuda-drivers
    Version: 520.61.05-1
    Priority: optional
    Section: multiverse/devel
    Maintainer: cudatools <cudatools@nvidia.com>
    Installed-Size: 7,168 B
    Depends: cuda-drivers-520 (= 520.61.05-1)

The solution is to install the latest CUDA driver (which is always 100% backward compatible) but have multiple version of the libraries in /usr/local. Granted it can get messy with various package manager doing their own things.

Usually the CUDA versions end up uninstalling other CUDA versions and drivers due to package manager issues.

I wish nvidia would just release a `sudo apt install cuda-all` that just keeps you updated with ALL possible cuda versions simultaneously. I know it would be 20GB but that's fine, it's a drop in the bucket compared to the datasets I play with.

If you use the run files rather than the package manager files it's always installed in completely separate folders inside /use/local/cuda.

What you're describing with an "all" install can somewhat be accomplished with containers right now and none of the dependency problems.

But containers, X forwarding, visualization, and OpenGL is a complete shitshow. I have apps that work outside of a container but freeze the UI that the mouse pointer won't even work when run from within a container.

+1 for runfiles but beware of the multi-gigabyte installer + installed footprint. Not complaining per-se, just plan for it.

Problem with runfiles is I've run into other .deb packages that I'd like to use that are marked as dependent on 'cuda'.

Using a runfile is ditching the package manager on a package-managed system, instead of using the package manager correctly.

> PyTorch did it right, it now ships with its own CUDA and doesn't take a shit about version what you have in /usr/local

Is that even legal ?

Up to my understanding Cuda is covered by an EULA license that explicitly require end user agreement.

"Meh" who cares. It works.

> now ships with its own CUDA and doesn't take a shit about version

Bundling system dependency in python software is always a giant shit show. I would not name that "Everything should do the same"

Any other python library (randomly cuPy) that also ship its own Cuda, with its own different version, will segfault happily because you have duplicated symbols in a single process.

So randomize the symbols with appeneded UUIDs

What you just quote is more or less what the GNU glibc is doing with symbol versioning which is an other example of giant shit show and an ABI nightmare.

And:

(1) It is a solution only Nvidia can provide: Cuda is a binary. And they will certainly not do that just to please the python community.

(2) That just create an other set of problems just because python packaging sucks in the first place.

Lets just solve the initial problem, shall we ?

That's right, we've tested down to pascal, but this should work on Kepler too since CUDA and the underlying libraries support it.

I have a spare Titan Z that could be useful if so.

Will you publish benchmarks for e.g. K80? Or provide a way for users to contribute? It's really handy to know, e.g. comparable to what is Resnet50 inference on a bunch of architectures.

Hi, what specifically are you looking to benchmark on the K80? Users are free to contribute and we've had many external PRs.

Contribution guide is here: https://github.com/NVIDIA/MatX/blob/main/CONTRIBUTING.md

well the root README.md has some "benchmarks" at the very top. maybe there's no existing benchmarks doc with more details? or even a repo of what's in the README? it's like if numpy is 5 sec on CPU, but a K80 is only 100ms, then the K80 cost might be worth not going for the A100 at 3ms. Similar argument for jetson.

We have benchmarks in the benchmarks directory, but these are for things like convolution, matrix multiples, etc. It's not for running a traditional benchmark set like resnet.

Like most benchmarks it really depends on what you want to do, and since it's a general library everyone might care about different things.

I saw those, yes having the code is great but I’m more interested in the actual numbers. E.g. what does this sample do on A100, P100, T4, K80, Jetson nano? The analog is things like Resnet50 get tested and reported and then you know if Resnet50 might work for your budget / hardware.

Versus the advert in the root Readme, which is impressive but gives no data on the pareto.

Hi, if you don't mind opening an issue asking for this we can run these and put in the readme.

Reply on news.ycombinator.com