DRAM Crunch: Lessons for System Design

I think it's hard to overstate how annoying working in a hardware-adjacent part of the industry has been in the past few years. We've only just recovered from the covid era supply chain issues, and it looks like we will continue to have shocks, after shocks, after shocks.

I recently bought a sample for a NUC-style board that contains an N355 processor, ram, and disk in a little kit. In the couple of weeks between when I put the PO in for the sample and the time it was delivered it had gone up in price from ~£600 to ~£1000. That might be a huge difference in BOM costs for a given product based on that design.

Real-terms prices for embedded computers must now be 2-4x what they were in 2020 for "similar" (i3, i5, i7, though I know there have been substantial improvements since then) positions in the market due to inflation and price gouging.

It's terribly difficult to keep up. This impacts far more than just AI. IoT devices, scientific instruments, kiosk systems etc have all skyrocketed in BOM costs because of this. Almost everything contains RAM, and a ~2x increase in an already expensive component might scupper a whole design.

This is what keeps Amodei and Altman up at night. Their whole moat is data centers. But what if we didn't need the data centers?

The weak link in the argument is the "1-2 GB DRAM stays stable" claim. That tier is stable because demand sits elsewhere. If AI workloads actually migrate down-capacity at scale (which is what the piece advocates), pricing pressure follows them.