Hacker News Reader

Comment by zaptheimpaler | Hacker News Reader

How about we move to external only GPUs with huge connectors? If GPUs are half the size, power consumption and price of a PC now, they might as well be a separate device. As a bonus the rest of the motherboard & PCs actually get much smaller. A PC without any spinning disks could conceivably just be the size of a NUC by default, something you can travel with when you don't need the beefy GPU.

View on HN

The most expensive part of GPU operation (power-wise) isn't computation, it's actually moving information around.

Moving the GPU further away from the CPU and storage is going to lead to worse latency and power requirements.

Over the next 10-15 years, we'll probably see CPU+GPU packages become mainstream instead of CPU (with basic integrated graphics) and a separate GPU.

Given that every Mac, AMD-based XBox and Playstation, and a chunk of the Ryzen market are shipping decent unified CPU/GPU combos, I would say that day is mostly here.

> Over the next 10-15 years, we'll probably see CPU+GPU packages become mainstream instead of CPU (with basic integrated graphics) and a separate GPU.

its already the case with arm socs.

But home PC users / high end GPU buyers (rightly or wrongly) are fanatical about modularity. The server market will likely (has already?) shifted.

I’m inclined to say that’s more due to integrated gpus not being a good replacement if you need CUDA or play games; If a better combined alternative existed (and not at an outrageous price) I don’t see the majority skipping it die to lack of being modular.

The most expensive part of GPU operation (power-wise) isn't computation, it's actually moving information around

I doubt that. Compare the GPU temperature (a good proxy for power consumption) when playing a game or doing GPGPU stuff vs playing a video (without GPU acceleration, so it's just acting as a framebuffer). The former involves far more computation, and gets the GPU much hotter.

No. There was an internal nvidia presentation from a few years ago that stated that moving data was the hard part. (I can't find the presentation any more, but if anyone can find it, please post it below.)

Previously graphics cards were essentially designed with a single area of the card handling computation, and another area holding memory. Data would need to be moved from the memory, to the computation area, then back again if there were changes that needed to be stored.

As the computation and the memory demands became larger, those areas had to handle more, but so did the bus between those two areas. What was a negligible overhead for the bus became more pronounced as it had to handle more data.

Eventually the energy overhead of transporting that much data across that distance started constraining what was possible with graphics cards.

That is why graphics card architectures have shifted over time to place memory cache units next to computation units. The less distance the data needs to travel, the smaller the power requirements. It's also led to the investment and adoption of stacked memory dies (why grab data from 5cm away in the x-y plane when we can stack memory and grab data 5mm away in the z-direction).

Moving around data is indeed a major issue for any throughout oriented device. But for a gaming GPU, PCIe BW has never been an issue in any of the benchmarks that I’ve seen. (Those benchmarks artificially reduce the number of PCIe lanes.)

In fact, the 4000 series still has PCIe 4.

Moving data around for a GPU is about feeding the shader cores by the memory system. PCIe is way too slow to make that happen. That’s why a GPU has gigabytes of local RAM.

I think what the OP is suggesting is that PCs should have external PCIE connectors for plugging in GPU modules.

The problem with external gpus is that the pcie5x16 slot requires signal integrity chips every 5 inches or so [1]. Even for pcie4 it's bad, many people had blue screens or boot issues when using pcie3 only rated riser cables with a pcie4 video card, even though electrically they have the same connections. So, having a huge cable with x16 lanes of pcie5 on the back of the computer doesn't seem that feasible. Maybe fiber optics could be a thing.

[1] https://www.planetanalog.com/signal-integrity-challenges-set...

Fiber optic would have higher latency vs. copper. Not sure how much of a difference that would make on a bus but I would assume the timing is pretty tight.

I remember linuxtechtips using a meter long wire to connect a 3090 to a computer and it worked flawlessly.

3 metres, not to mention it wasn't a single cable, but a bunch hooked up to each other. Though I believe that was PCIe 3, so 4 might be more picky.

You can easily buy a NUC ECE (size of a graphics card) and use a pcie cable to connect to an actual card. This is my plan for my next machine.

How would that work? I admit I never even thought of PCIe cables, thought it was always just a physical slot you have to put stuff in.

there are PCIe riser cables that you can imagine as an extension cord for a PCIe slot. they're popular in some thin server chassis or in miniITX builds where the form factor dictates the GPU must be in a certain orientation to fit.

there's also the method of external GPUs via Thunderbolt (which carries PCIe) that is more practical for putting a GPU in an entirely separate enclosure

Ok so just plug your computer into the GPU ;)

I wonder how far that connection could be before it stops working. Probably not much!

Here is an LTT video about how many PCI-E extensions one can daisychain without problems.

The limit is around 3 meters.

https://www.youtube.com/watch?v=q5xvwPa3r7M

One should note that it will works way worse with higher pcie versions of today

Are you referring to pcie 4?

The RTX 4000 series uses PCIe 5, and the article references PCIe 6.

I was wondering about that. According to Nvidia RTX 4090 product page, it still used PCIe 4.

Yup, it only uses PCI-E5 power connectors. The GPU is gen4.

A meter of extension adds ~10ns of latency (speed of light).

Isn't this what the thunderbolt enclosures with GPUs are trying to do?

I’m pretty sure Thunderbolt only has the equivalent of 4 lanes of PCIe 3.0 while GPU cards tend to be 16 lane.

Reply on news.ycombinator.com