Hacker News Reader

Comment by thethirdone | Hacker News Reader

I get that, but given that GPUs currently can't use (for physical connector) / don't benefit from more than 16 lanes. Unless you are using multiple GPUs or another PCI device, you would not benefit from more than 16 lanes. How does that hold GPU manufacturers back?

View on HN

That sounds suspiciously like a chicken and egg problem. If the dominant CPU could only use 16 lanes, it makes sense to not put too much effort into supporting more if they can't be used effectively (other than being able to point at Intel as the problem and thus piss Intel off).

I don't think it is a chicken and egg problem. There are several of CPUs that support > 16 lanes, so a high-end GPU (paired with a high end CPU) could use more lanes if it needed it.

Currently it seems 8 lanes of PCIe 3.0 is not quite enough, but 16 lanes is plenty. So we'll need a few years to get up to utilizing 16 lanes completely.

At which point 4.0 may be ready.

> Unless you are using multiple GPUs or another PCI device

You mean like having a GPU and one or more PCIE NVMe storage-devices?

No, as in 4 GPUS and/or one or more PCIE NVMe storage devices or PCI based comms cards such as infiniband cards.

This. I've been trying to put together a small home lab xeon e3 system that can theoretically support nvme+gpu+infiniband. It's tough.

Typically you get something like 3 full length PCIe slotson a xeon e3 board (c236 chipset). BUT they can't all run full speed at 16x (16/16/16). You have to then choose between 16/8/8 and 16/16/0 due to the lack of PCI lanes.

Can you explain why you need 16/16/16? Wikipedia tells me that one x16 PCIe 4.0 connection has a capacity of 32GB/sec (x8 is 16GB/sec). A quick search shows NVMe drives with a sustained read or write speed of ~2-4 GB/sec. Equally for the GPU - you need to move a lot of data in/out of the GPU to saturate a x8 connection.

It's rare to run that sustained, usually it's bursty so the extra bandwidth is very much needed.

Some parallel computing problems are best solved by one CPU and a lot of GPUs. In some of these problems, inter-GPU communication is important. If you have each GPU using a single PCI lane, then the compute power can be bound by iner-GPU communication bandwidth.

Yeah, there are definitely cases for threadripper's PCI lanes. But a single GPU is not one of those cases.

16 lanes is required unless you are ready to give up 20% gaming performance on latest games. see the side by side comparison below.

https://www.youtube.com/watch?time_continue=2&v=8KOTiee5RqA

on top of that, many people want to stream their game video when they play their games on full speed, then a couple NVMe SSDs eat those lanes as well.

But if you're dropping 50% of the lanes and only getting a 20% speed drop, then it looks like you're not using those 16 lanes to capacity. Logically you'd expect closer to a 50% performance drop.

each Intel upgrade gives you 5-10% performance improvement. 20% is like 3 generations of Intel "innovation".

What I'm saying is that if the video card you were using had 16 lanes available and got X performance. Then you took away half the lanes available to graphics card, so now you have 8 lanes available, but performance has only decreased by 20%. Meaning that the game or application was only using a fraction of those extra lanes. So basically it wasn't fully exploiting the hardware available.

Second graphics cards are not that unusual, and by the time you then want an NVMe drive too, you're bumping up against limits.

More is definitely better here.

Reply on news.ycombinator.com