Hacker News Reader

Comment by amelius | Hacker News Reader

The SSDs are necessary because Apple's architecture doesn't allow RAM upgrades. Reminds me of someone who said "640KB ought to be enough for anybody".

View on HN

To be fair, nobody has upgradable memory in any system that has enough memory bandwidth and compute power to run LLMs with decent performance. It might be interesting to compare against some decade-old x86 server or workstation stuffed full of LRDIMMs to reach 1.5–2TB of RAM, but the bandwidth would be only slightly faster than a desktop today with high-end DDR5: nowhere close to GPU bandwidth. So performance would still suck.

Designing for extreme expandability comes with pretty steep tradeoffs.

Take a look at AMD’s 12-channel memory servers. The newer Epycs are up to 16-channels now, 1.6TB/s. Pretty great for inference.

Sure, if you want to make a comparison where the price tags aren't the same order of magnitude, then a recent server is obviously going to be powerful. But since the baseline of this comparison is a laptop and several Thunderbolt SSDs, the kind of servers or workstations with 1.5–2TB of RAM that you can reasonably compare against would have to be the really old ones, barely new enough to support that much total RAM.

And despite the theoretically high memory bandwidth of recent EPYC CPUs, approximately nobody who can afford one is doing LLM inference on them.

You would have the same issue with a GPU. I have 8 slots on my CPU but it cant handle 2.8TB of ram. I cant even buy 2TB for it because even though the mobo is rated for it, nobody ever made those chips (someone will sell me 8x256gb 8Rx4 for $15k but there’s no way the memory controller can handle that). 1TB is $10k and forced it to run slower.

Apple uses SoC's which means the RAM is simply not upgradable because it is a SYSTEM on a CHIP

Typically system on a chip does not include DDR. There are differences in the fab process between DRAM and SoC/SRAM.

Gates never said that, for what it's worth.

> In an April 1985 InfoWorld editorial, James Fawcette wrote that Gates had said something like: “When we set the upper limit of PC-DOS at 640K, we thought nobody would ever need that much memory.”

So yes, Bill Gates denies that story. So it just depends on who you believe.

But to flat out say he never said it is too confident.

I was mostly curious about the source and figured that would bring it out. Sorry.

Reply on news.ycombinator.com