Hacker News Reader

HuggingFace says this model has 753B parameters, which will need a lot more RAM than a maxed-out MacBook Pro. With 40B active parameters, running from SSD would need patience.

View on HN

I’ve wondered for a while if anyone is working on very wide channel parallel (kind of like RAID 0) SSD for this purpose. Couple that with a tensor processor and that would be interesting.

There's talks about HBF. F for Flash -- HBM packaging and bus width, but using NAND memory.

e.g. https://www.sandisk.com/company/newsroom/press-releases/2026...

For an fp4 quantization it should fit with room to spare for KVCache

Aren't Macbooks limited to 128GB RAM?

FP4 would require >350GB RAM + KV cache, so no.

Oops, you’re right. My brain understood Mac Studio for some reason.

Reply on news.ycombinator.com