HBM’s Future: Necessary but Expensive

The article doesn't sell it well. Basically, its fundamentally more expensive... But lol, where else are you gonna go for your reticle size chips?

Its more power efficient than GDDRX, but not necessarily far from LPDDRX.

As a they speculated, a compromise seems like the future. Like a wider memory standard with stricter trace requirements, but not wide enough to require an interposer + control chip.

Or maybe just external memory controller/cache chips like the 7900 XTX, where the chip space for all the pins is less of a concern.

I would be very surprised if AMD isn't looking into "HBM-lite", i.e. same 2.5D structure, but instead of the parallel HBM interface, using a serial IF/GMI interface (fewer pins, higher frequency), so that you can mount it on a much cheaper organic (PCB) interposer.

HBM is getting more expensive AND we've hit the SRAM density scaling limits AND current GPU lines shipments delays are expected to last until Q2 of next year so no price relief anytime soon. Ouch.

I wish the author would have put some actual numbers on the cost of the different technologies. Arguing about tradeoffs including that information would be much more meaningful.

Perhaps I'm wrong... but as far as I can see, no RAM technology currently allows variable latency?

Ie. the host says "I want data from this address", and the RAM can reply 2 clock cycles later with "Here is that data you asked for" or "I'm still getting that data for you, just wait a few more clock cycles please".

Such schemes allow the RAM manufacturer to implement error detection/correction/remapping, which would allow them to push their designs far closer to what physics allows. (you can make something much much smaller/faster if you only need it to work 99% of the time, and use a fallback path/mechanism for the remaining 1%)

> HBM

> Rambus

I'm not really sure I believe anything they say... I had one of those early Pentium 4s with RIMM memory, and it was one of the hottest and slowest memory I have ever worked with. DDR4/5 seems to work for me just fine.

Wouldn't it be cheaper to integrate DDR chiplets in the package, like Apple is doing? Even though the approach is not competitive with HBM in performance, on Apple processors it does seem to offer better power efficiency and bandwidth compared to off-package memory. But for some reason neither Intel nor AMD are moving in that direction.

So who is using it, and where? I saw AI mentioned. Is this for the datacenter or more for the graphics/AI workstation?

Tangential question: is Oxide using HBM in their racks? My understanding is that part of their value proposition is making hardware tech that's usually built for the hyperscalers available to on-prem datacenters; this seems like the kind of technology they might be referring to.

At the HBM neuroscience conference, amd I agree with the headline

demand always cheapens production cost