Hacker News Reader

Comment by WithinReason | Hacker News Reader

Looks like good value, but I wonder if it would get CPU/RAM bottlenecked, especially if you want to train something with a lot of preprocessing in the pipeline. Something comparable I've found with 7x4090 which comes to about $50k, but with much better CPU/RAM (3x CPU, 4x RAM, 5x SSD):

https://www.overclockers.co.uk/8pack-supernova-mk3-amd-ryzen...

View on HN

> 6x PCIe 4.0 x16 (64 GB/s)

Wikipedia [0] states that PCIe 4.0 x16 has a throughput of ~32GB/s, what does the (64 GB/s) indicate on the website, is this just a typo and you have 6x ~32GB/s or does it mean in total you can "only" expect a throughput of 64GB/s all lanes slots combined?

If so, wouldn't you also be bottlenecked by the PCIe bandwidth (when moving data between CPU and GPU)?

[0] https://en.wikipedia.org/wiki/PCI_Express#Comparison_table

Well they're specifying the AMD EPYC and one of the things that the server line of AMD CPUs do that the consumer grade ones don't, is they have lots of connectivity. So for example an AMD EPYC 8324P is a 32 core CPU with 96 lanes of PCI Gen 5. Given that the 4090 GPU is PCI Gen 4, I think that's where you get the discrepancy. The 6 GPUS are connected in parallel to the CPU with 6 x16 connections (96 total lanes), the CPU could do this at Gen 5 (64GBs for each GPU) but the 4090 GPU is Gen4 only, so you'll only actually get 32GBps per connection.

Most EPYCs have 128 PCIe lanes, so I'd expect a full x16 link for all six GPUs.

Pedantically, the combined bidirectional bandwidth of PCIe x16 is ~64 GB/s, as it's a full-duplex ~32 GB/s link, but that's an awfully misleading spec if this is the intent (akin to claiming Gigabit Ethernet is 2 Gb/sec).

It's the same way NVIDIA states bandwidth for PCIe and NVLink.

https://www.nvidia.com/en-us/data-center/h100/

It's 32GB/s in both directions. So when exchanging data two GPUs each can do this at 64GB/s. Is that a useful way to measure it? Who knows.

Closer to $42k, i think, at least if you're comparing it to the Tinybox price -- the price in pounds on the site includes VAT, which you wouldn't pay as a business or if you were getting it for export outside the UK, whereas you'd need to add on VAT if you were getting a Tinybox in the UK.

It’s weird how non-specific the CPU is there. Why wouldn’t they list a CPU part number? We don’t even know what generation of Epyc it is. (I get that it’s not the focus… but it is still important.)

Reply on news.ycombinator.com