Hacker News Reader

Comment by TwoNineA | Hacker News Reader

I hope for a little more PCIe lanes so I can run 2 gaming VMs on these and upgrade my old Threadripper.

There is fuck all difference between x8 and x16 for gaming. Heck with PCIe5 even dropping to x4 is borderline noticeable outside of benchmarks.

100% this

The PCI-Express bus is actually rather slow. Only ~63 GB/s, even with PCIe 5 x16!

PCIe is simply not a bottleneck for gaming. All the textures and models are loaded into the GPU once, when the game loads, then re-used from VRAM for every frame. Otherwise, a scene with a lowly 2 GB of assets would cap out at only ~30 fps.

Which is funny to think about historically. I remember when AGP first came out, and it was advertised as making it so GPUs wouldn't need tons of memory, only enough for the frame buffers, and that they would stream texture data across AGP. Well, the demands for bandwidth couldn't keep up. And now, even if the port itself was fast enough, the system RAM wouldn't be. DDR5-6400 running in dual-channel mode is only ~102 GB/s. On the flip side the RTX 5050, a current-gen budget card, has over 3x that at 320 GB/s, and on the top end, the RTX 5090 is 1.8 TB/s.

You are correct that games generally are not PCIe limited. But you are incorrect that games just upload everything ones and be done. Most modern engines are most certainly streaming in and out assets all the time.

> All the textures and models are loaded into the GPU once, when the game loads, then re-used from VRAM for every frame. Otherwise, a scene with a lowly 2 GB of assets would cap out at only ~30 fps.

Ah, not really these days, textures are loaded in/out on demand, at multiple different mipmap levels, same with model geometry and LOD's. There is texture and mesh data frequently being cached in and out during gameplay.

Not arguing with your points around bus speeds, and I suspect you knew the above and were simplyifing anyway.

The biggest difference for me for PCIe 5.0 has been additional bandwidth for my M2 drive.

Faster M.2 drives are great, but you know what would be even greater? More M.2 drives.

I wish it was possible to put several M.2 drives in a system and RAID them all up, like you can with SATA drives on any above-average motherboard. Even a single lane of PCIe 5.0 would be more than enough for each of those drives, because each drive won't need to work as hard. Less overheating, more redundancy, and cheaper than getting a small number of super fast high capacity drives. Alas, most mobos only seem to hand out lanes in multiples of 4.

Maybe one day we'll have so many PCIe lanes that we can hand them out like candy to a dozen storage devices and have some left to power a decent GPU. Still, it feels wasteful.

The M.2 form factor isn't that conducive to having lots of them, since they're on the board and need large connectors and physical standoffs. They're also a pain in the ass to install because they lie flat, close to the board, so you're likely to have to remove a bunch of shit to get to them. This is why I've never cared about and mostly hated every "tool-less" M.2 latching mechanism cooked up by the motherboard manufacturers: I already have a screwdriver because I needed to remove my GPU and my ethernet card and the stupid motherboard "armor" to even get at the damn slots.

SATA was a cabling nightmare, sure, but cables let you relocate bulk somewhere else in the case, so you can bunch all the connectors up on the board.

Frankly, given that most advertised M.2 speeds are not sustained or even hit most of the time, I could deal with some slower speeds due to cable length if it meant I could mount my SSDs anywhere but underneath my triple slot GPU.

> I could deal with some slower speeds due to cable length

Observing server mainboards reveals many PCIe 5.0 connectors for cables to attach PCIe-SSDs looking similar to SATA ones.

Agree that M.2 is fiddly. PCIE cards with M.2 sockets are nice nice for desktops and servers, then one can just unplug it to do operations.

Asus m.2 add in risers would like a word ala Gene X670E.

There are add-in cards with PCIe switch chips that will let you put a large number of drives into a single PCIe slot.

Including ones that have controllers, if your motherboard doesn't have enough lanes or it doesn't support bifurcation. I have a Rocket 7608A, which gives you 8 M.2 slots in a PCIe 5.0 x16 card: https://www.highpoint-tech.com/nvme-raid-aic/gen5/rocket-760...

> Alas, most mobos only seem to hand out lanes in multiples of 4.

AFAIK, the cpu lanes can't be broken up beyond x4; it's a limitation of the pci-e root complex. The Promontory 21 chipset that is mainstream for AM5 does two more x4 and four choose sata or pci-e x1. I don't think you can bifurcate those x4s, but you might be able to aggregate two or four of the x1s. And you can daisy chain a second Prom21 chipset to net one more x4 and another 4 x1.

Of course, it's pretty typical for a motherboard to use some of those lanes for onboard network and what nots. Nobody sells a bare minimum board with an x16 slot, two cpu based x4 slots, two chipset x4 slots, and four chipset x1 slots and no onboard perhipherals, only the USB from the cpu and chipset. Or if they do, it's not sold in US stores anyway.

If pci-e switches weren't so expensive, you might see boards with more slots behind a switch (which the chipsets kind of are, but...)

Your comment is basically the "tl;dr" of this Techpowerup article (which is great and people should read it if they are unconvinced or curious): https://www.techpowerup.com/review/nvidia-geforce-rtx-5090-p...

Main problem seems to be they're kinda badly utilized (IMHO) on many motherboards. Most seem to go with two x16 slots so you get x8 lanes in both.

There are some exceptions, but I haven't seen one with for example four x16 slots that support PCIe 5.0 x4 lanes with bifurcation.

You can buy add-in cards that do lane bifurcation

E.g. https://www.ebay.co.uk/itm/126656188922

Most motherboards don’t go beyond 2x8 with 2x16 physical slots because there is little actual use for it and it costs quite a bit of money.

Bifurcation is done on the CPU.

Asus 5.0 Hyper M.2 with yield 4x4x4x4 in a board that is setup for bifurcation of PCIE slot 1 like that.

I have run 4 GPUs this way and it works very well.

You're not getting more lanes without a new socket. Or a PCIe switch, which is expensive.

This. I needed a high speed link between two PCs and bought a mellanox card, cue me surprised that a consumers PCs do not have enough PCIe lanes to handle both a thickboii GPU and a thickboii 200GBe mellanox card...

for that you need new socket and motherboard. you need to physically route those extra lanes to pcie slots or other components

And even when AMD does move their mainstream desktop processors to a new socket, there's very little reason to expect them to be trying to accommodate multi-GPU setups. SLI and Crossfire are dead, multi-GPU gaming isn't coming back for the foreseeable future, so multi-GPU is more or less a purely workstation/server feature at this point. They're not going to increase the cost of their mainstream platform for the sole purpose of cannibalizing Threadripper sales.

Myltigpu is alive, well, and more effective than ever.

Lossless scaling across 2 8x slots rocks.

Had to look up what vm gaming is. What's your motivation? If you don't mind sharing.

Reply on news.ycombinator.com