Hacker News Reader

Comment by mkl | Hacker News Reader

I'm trying to work out if I should buy a 48GB M4 Pro Mac Mini now, or wait for M5 Pro ones later this year. For AI/ML purposes, mostly. As far as I can tell, the new M5 MacBooks didn't go up much or any for the same amount of RAM?

View on HN

I wouldn't buy a local machine for AI/ML purposes unless you have an actual defined use case and programs to run (perhaps even being able to test them at an Apple Store).

Otherwise you may end up like others using a high-spec Mac mini to just access online models.

I do not appreciate you calling me out personally!

Which one are you, Mac or Mini?

The AMD Strix Halo pc's are another option. I was also debating between a mac mini but decided to go the AMD route.

I'm eyeing that seriously too. Are you running linux on it by chance? Would love to hear from someone running linux on a non-Apple AI capable machine

At this point is the performance advantage of Apple CPUs even worth it if you can't upgrade the ram itself? I'm thinking you might be better off building a PC and putting the absolute bare minimum RAM in it, with plans to swap that out with good stuff in a year or two once the RAM market stops being insane.

are we sure the RAM market will stop being insane in a year or two or could this be the new norm?

Why should it be the new norm? We have an abnormal situation now, of massive amounts of investor money being poured into unprofitable bets, that this time had the side effect of eating up hardware components. There are two possible outcomes:

1. Yes, it's the new normal, then production capacity will be increased and prices fall.

2. No, it's not the new normal, the bubble pops and component prices come crashing down when buyers default etc.

Option 2 has been the normal outcome of these situations so far. But sure, questions remains how long all of this will take.

Option 3: the global wars increase and continue to be the new normal with shipping routes disturbed until the climax, china annexes Taiwan.

In that case prices will continue to rise (among other things).

I don't know if it'll be a year or two, hard to say exactly when the AI bubble will pop, but I feel quite certain it's coming. The AI stuff is great but most of the money being thrown around to all these different companies is mostly going to be wasted. Investors don't know who the winners and losers will be, just like when people were investing in pets.com instead of amazon.com.

Every ten years the RAM cartel raises prices (it's not really about AI, see Gamer's Nexus) and every ten years it is forced to lower them again.

It's the RAM bandwidth that's the advantage, 300GB/s for M5 Pro. RAM in slots is way slower, ~50GB/s.

But for ML workloads the comparison isn't between slotted CPU RAM and Apple's unified RAM, it's between Apple's unified RAM and dedicated GPU VRAM, which can more than double even the M3 Ultras bandwidth at up to 1.8TB/sec. Apple Silicon makes a unique set of trade-offs that shine in certain areas but they are still trade-offs nonetheless, so it really depends on what exactly you're doing with the hardware.

Dedicated GPU VRAM is much scarcer than the unified RAM you get on Mac platforms. This is a big deal for SOTA LLMs that combine high memory footprint with a need for high memory bandwidth in order to get acceptable performance.

RAM, disk, CPU, GPU, for me it isn't for quite some time, then again I have been mostly a Windows/UNX person, only using Apple gear when assigned via project delivery.

I don't expect the ram situation to get any better soon if that's what you're asking.

Depends what kind of AI/ML purposes you are intending to work on

Reply on news.ycombinator.com