Hacker News Reader

Comment by codedokode | Hacker News Reader

32 Gb DDR4 RAM module has a bandwidth of 25 Gb/s and costs $160. If you buy 8 of these, you get 256 Gb RAM with 200 Gb/s bandwidth at $1280. And if you buy 16 x 16 Gb modules (each at $60) then you can get 400 Gb/s of bandwidth for $960.

The only problem, you need 8 or 16 memory controllers. Memory controllers are not that expensive: Intel Core i3-14100F has 2 channel controller and costs $110, so we can estimate that 16-channel controller should cost not more than $880, and 8-channel controller should cost $440.

So isn't it better to make a cheap CPU with 16 DRAM controllers instead of this $4K gear having only 128 Gb? Or maybe 2 CPUs each having 8 RAM channels?

DDR5 costs 2 times more ($360 for 32 Gb) while not even having 2 times the bandwidth so it is not worth buying. It is more reasonable to make more RAM channels and stuff them with DDR4.

View on HN

If you want Epyc go for it. The motherboards can be quite expensive though.

So what I am trying to say, industry took a wrong turn. Instead of moving to over-priced DDR5, they should just make even cheapest CPUs support 8/16 DDR4 channels. Because a 32Gb DDR5-4800 module costs $360, and two 32Gb DDR4-3200 modules cost $320, so you get twice more size, more bandwidth and it costs you less. DDR5 is just a rip off.

Each memory controller interface is a not-insignificant number of PCB traces. Increasing the number of memory controllers may dramatically increase the number of PCB layers (or may not, it really depends on the CPU pinout) but it definitely will increase the number of pins on the CPU socket.

This is one of the main reasons (the other is the number of PCIe lanes) why high end desktop and server CPUs have like double the number of pins and so much bigger sockets as compared to consumer desktop CPUs.

Then what's about using 4-8 cheapest motherboards with 64Gb DDR4 and a cheap CPU, and connecting them via PCIE x16 sockets?

And as for DRAM channels, typical cheap motherboard has 2 channels and 4 slots, it should not be super difficult to add 2 more channels.

Don't you need a new CPU to support more channels? It doesn't sound like a cheap, trivial change to upgrade my Ryzen to a Threadripper.

To get memory mapping across systems you'd want to look into infiniband (https://en.wikipedia.org/wiki/InfiniBand) but the adapters generally aren't cheap. Common to find in super computer clusters.

If you have a desktop CPU with 2 memory channels and 2 DIMMs per channel, then the 2 DIMMs on each single channel share all of the address and data lines, there's just a chip select difference to pick which DIMM you're addressing. There is some extra loading on the lines, since you have 2 DIMMs, hence the supported speeds are usually slightly lower, but you only add 1-2 more actual PCB traces to have 2 DIMMs per channel vs 1 DIMM per channel. In contrast, adding another memory controller would add upwards of almost 100 additional PCB traces from the CPU.

> just make even cheapest CPUs support 8/16 DDR4 channels

Isn't adding pins kind of expensive?

Well to be honest, there are a lot of NOOP pins on CPUs, but using them basically means fabbing a new die altogether, which is basically making a whole new CPU altogether.

This is an odd comment. Pins are seriously expensive. Companies don't just throw them in for fun. Usually they're necessary for signal integrity or packaging constraints.

Yes but not $1000 level expensive, right? It is just a tiny piece of metal, and on laptops there are no pins at all.

I think this is about semiconductor design and/or die packaging related difficulties/costs. idk exactly why, but "pin count" in this exact expression is not rare at all on chip marketing materials.

In the alternate reality where this happened, wouldn't the price of DDR4 still be sky high? We'll ignore any costs for CPU, chip set, and motherboard redesign. You're just pushing the demand somewhere else.

> any costs for CPU, chip set, and motherboard redesign.

Yes they spent those costs to switch from DDR4 to over-priced DDR5 and I suggested the cost could be spent on adding more DDR4 channels instead.

The problem is if DDR4 was viable here it would also be absurdly expensive. There’s no world in which we survive this demand without a corresponding increase in supply. It’s not like there are zettabytes of spare cheap DDR4 sitting around.

> even cheapest CPUs support 8/16 DDR4 channels

In addition to cost and possibly market segmentation, presumably the additional power consumption wouldn't be worth the tradeoff for laptop parts. If you want high channel count you've always been able to purchase power hungry datacenter gear. You can also pick up surplus 10 GbE or even 100 GbE fiber NICs to link up your beowulf cluster. You'll probably have to run a few new electrical circuits and a small bit of HVAC if you're installing this in a residential home but it's entirely doable to cram 10 kW in a closet if you feel like it.

Alternatively you could just rent cloud instances by the hour to avoid the hassle.

Reply on news.ycombinator.com