Hacker News Reader

Comment by gautamcgoel | Hacker News Reader

Did I read the article correctly? Is there 32GB of DRAM on-chip? Or was the DRAM just standard DIMMs?

Not on-chip no, it's still DDR5 DIMMs. What is cool about is that it is using custom DIMMs and memory controllers to do 8B-granularity accesses. Plus, each core has its own MC, as you can see on the die shot. If each MC manages 4GB of DRAM, that would explain the 32GB per chip.

Can you elaborate on the granularity? Why is that important?

Your regular DRAM memory controller will use the 64B bus to pull in a single 64B line of memory. In your modern x86 systems, that's equal to a cache line.

If you are only accessing a single 4B or 8B element of that cache line >80% of the time, as shown on the slides, you are wasting 7/8th of all memory bandwidth with irrelevant data. If you were to use that 64B memory bus to access 8x different 8B memory locations, you get a large boost to your effective bandwidth.

[edit: I think I'm wrong!] I'm pretty sure this is HBM (High Bandwidth Memory) on the same substrate: https://en.wikipedia.org/wiki/High_Bandwidth_Memory

There are some other explainers out there that go into more detail, but Nvidia is also doing this with Grace Hopper (I think?), and Apple with their M-series chips.

Update: It looks like it's just DDR5 ("custom DDR5-4400 DRAM") in this case.

Apple is not using HBM on M-Series chips, although I bet the speed benefit would be massive.

It's regular DDR5.

Reply on news.ycombinator.com