Hacker News Reader

Comment by mikkupikku | Hacker News Reader

At this point is the performance advantage of Apple CPUs even worth it if you can't upgrade the ram itself? I'm thinking you might be better off building a PC and putting the absolute bare minimum RAM in it, with plans to swap that out with good stuff in a year or two once the RAM market stops being insane.

View on HN

It's the RAM bandwidth that's the advantage, 300GB/s for M5 Pro. RAM in slots is way slower, ~50GB/s.

But for ML workloads the comparison isn't between slotted CPU RAM and Apple's unified RAM, it's between Apple's unified RAM and dedicated GPU VRAM, which can more than double even the M3 Ultras bandwidth at up to 1.8TB/sec. Apple Silicon makes a unique set of trade-offs that shine in certain areas but they are still trade-offs nonetheless, so it really depends on what exactly you're doing with the hardware.

Dedicated GPU VRAM is much scarcer than the unified RAM you get on Mac platforms. This is a big deal for SOTA LLMs that combine high memory footprint with a need for high memory bandwidth in order to get acceptable performance.

are we sure the RAM market will stop being insane in a year or two or could this be the new norm?

Why should it be the new norm? We have an abnormal situation now, of massive amounts of investor money being poured into unprofitable bets, that this time had the side effect of eating up hardware components. There are two possible outcomes:

1. Yes, it's the new normal, then production capacity will be increased and prices fall.

2. No, it's not the new normal, the bubble pops and component prices come crashing down when buyers default etc.

Option 2 has been the normal outcome of these situations so far. But sure, questions remains how long all of this will take.

Option 3: the global wars increase and continue to be the new normal with shipping routes disturbed until the climax, china annexes Taiwan.

In that case prices will continue to rise (among other things).

I don't know if it'll be a year or two, hard to say exactly when the AI bubble will pop, but I feel quite certain it's coming. The AI stuff is great but most of the money being thrown around to all these different companies is mostly going to be wasted. Investors don't know who the winners and losers will be, just like when people were investing in pets.com instead of amazon.com.

Every ten years the RAM cartel raises prices (it's not really about AI, see Gamer's Nexus) and every ten years it is forced to lower them again.

RAM, disk, CPU, GPU, for me it isn't for quite some time, then again I have been mostly a Windows/UNX person, only using Apple gear when assigned via project delivery.

Reply on news.ycombinator.com