Hacker News Reader

Comment by temp0826 | Hacker News Reader

Probably a good thing SLI fell out of fashion. No consumer boards with multiple 16x, but a few with 2 8x (gated behind a "mode" switch). A few years ago it was looking like we were on our way to full 4 16x slots. For cuda/llm/whatever does it really matter if the cards are in 1x slots?

View on HN

It's the other way around. SLI falling out of fashion is why there are no consumer boards with multiple x16 slots. There's no longer any demand for it on the consumer side, so the CPU vendors only provide lots of PCIe lanes for expensive chips.

On the server side, seven x16 slot motherboards exist.

GPUs in 16x slots is still important for LLM stuff, especially multi-GPU, where lots of data needs to move between cards during computation.

Depends on what you're doing. I'm pretty sure the bandwidth for inference isn't much.

Depends, if it's tensor parallel or pipeline parallel. Only PP doesn't pass too much. TP does

A 16x PCIE 6.0 setup has more bandwidth than any dual channel DDR5 memory kit.

I would expect x8 at 5.0 speeds to be plenty for SLI. That's twice as fast as x16 slots were around the end of the SLI era.

... shouldn't the logic be opposite? "Bad that SLI went out of fashion, there's no way for two GPUs to communicate fast over pcie, and SLI would allow such fast bridge"

Whether or not SLI remained viable for gaming, Broadcom was going to jack up the prices on PCIe switches to the enterprise-only range. That's the real reason why consumer motherboards don't have more GPU slots. Mainstream consumer CPU sockets never had a wealth of PCIe lanes, there was just a brief span of years where PCIe switches were cheap so high-end consumer boards could offer several x8 or x16 slots (sharing bandwidth in ways that make diagrams like these important).

In previous decades, non-mainstream CPU sockets were also more accessible to consumer budgets; first-gen Threadripper started at only 8 cores, so it was possible to pay extra for more memory channels and IO lanes without also buying an excess of CPU cores. But that had little to do with the popularity or viability of multi-GPU consumer systems.

But PCIe switches are now more common than ever. How else do you think those high-end consumer boards are able to provide six M.2 slots?

Not all PCIe switches are made equal. IIRC, there are two kinds, and only the expensive one can work without adding excessive latency, important for GPUs.

NVMe traffic is very tame when compared to GPU traffic.

PCIe switches with current-generation link speeds and high lane counts are prohibitively expensive and have been absent from consumer motherboards since PCIe gen3 showed up.

The chipsets on consumer motherboards are pretty much PCIe switches plus some SATA and USB controllers, but they're clearly in a different league from anything that's relevant to connecting GPUs. The host interfaces are x4 or occasionally x8, and the downstream links don't support links wider than x4, and at most a few of those. The link speeds are often a generation (sometimes two) behind what the CPU's PCIe lanes support. The high-end motherboards for AMD's consumer platform support more SSD slots by daisy-chaining a second chipset off the first; you get more M.2 slots but it's all still sharing a single PCIe gen4 x4 link to the CPU.

In the PCIe gen2 era, it was common to see high-end consumer motherboards include a 48-lane PCIe switch to take x16 from the processor and fan it out to two x16 slots or some combination of x16 and x8 slots. That kind of connectivity has vanished from consumer motherboards, and isn't really common in the server or workstation markets, either. 48-lane and larger PCIe switches exist, but are mostly just used for servers to connect lots of NVMe SSDs.

Reply on news.ycombinator.com