Hacker News Reader

I'm super duper curious if there are ways to glob together VRAM between consumer-grade hardware to make this whole market more accessible to the common hacker?

View on HN

You can, for instance, connect two RTX 3090 with an NVLink bridge. That gives you 48 GB in total. The 4090 doesn't support NVLink anymore.

> The 4090 doesn't support NVLink anymore.

Are you sure about that?

that's what the press said: https://www.tomshardware.com/news/gigabyte-leaves-nvlink-tra...

You actually can split a model [0] onto multiple GPUs even without NVLink, just using the PCIe for the transfers.

Depending on the model the performance is sometimes not all that different. I believe for solely inference on some models the speed difference may barely be noticeable, where for other training activities it may make 10+% difference [1]

[0] https://pytorch.org/tutorials/intermediate/model_parallel_tu...

[1] https://huggingface.co/transformers/v4.9.2/performance.html

I remember reading about a guy who soldered 2GB VRAM modules on his 3060 12GB (replacing the 1GB modules) and was able to attain 24GB on that card. Or something along those lines.

Reply on news.ycombinator.com