Hacker News Reader

Hacker News

Comment by EagnaIonat | Hacker News Reader

Parent Full thread

EagnaIonat

Your statement is not really true any more.

I have an old M1 Max with 32GB that runs inference models fine, especially those using MLX.

Dedicated graphics cards are faster but not to the point that it matters.

That’s a six year old machine.

My newer machine M5 max 128GB will far outperform your typical 32GB gaming card once the model exceeds memory.

To even get close to that on a dedicated graphics card you are paying upwards of $20K.

That said, models are getting smaller and faster which allows me to run multiple different models with ease.

[edit] Just checked the 5 models I use take up 57GB when all loaded at the same time.

View on HN

> Dedicated graphics cards are faster but not to the point that it matters.

It definitely matters, if you're intending to run a Claude Code/OpenCode style agent workflow. Most of those harnesses start with 8-12k token contexts, which is a lot of prefill for a Mac but cheap for a CUDA GPU. The GPGPU compute on the fastest Macs is still trailing behind Nvidia's laptop GPUs; the highest-bandwidth Apple Silicon chip (now the M5 Ultra) has ~7x lower memory bandwidth than a single B100 card.

There's a good reason why Apple Silicon isn't to be found anywhere in the datacenter buildout. It's nonviable for training, and wastes electricity running real-world inference workloads.

Is it too much to give the harness a moment to load up the context when starting a new session? Is the brief pause truly a deal breaker?

That's my point as well. PC Graphics cards will be faster in a number of ways, but the Mac isn't slow for it to matter.

I dunno why Apple fans do this thing where they pretend that their specific workflow is the standard for how things work, and because they can do it so well on their Macs, that means Macs are the best.

To be specific, 5 models at 57 gb means you are using crap quantized models, which suck for any real agentic work. I mean, sure they give you some inference, but compared to the full parameter models like Qwen3.8 and Gemma4 that can run full agentic loops, you may as well just use cloud inference for the price.

You of course could "run" those larger models, but we both know that the tok/sec is dogshit on Macs for those.

And 57 gb is split across 3 cards quite easily, which will all be cheaper than your comparable Mac and way faster.

You really need to educated yourself on how running local models works and what the models like Gemma 4 are capable of, so you don't continue to waste money on Macs.

> I dunno why Apple fans

You mean someone who uses a Mac? I find the term "Apple fan" is used as a way to attack the person.

> that can run full agentic loops, you may as well just use cloud inference for the price.

You can run full agentic loops fine with quantised models. In fact that is likely what you are doing with a 32GB PC Graphics card. Or what model are you using?

You use the right model for the right job. For example granite4.2 is optimised for agentic work and only needs 5GB of memory.

Gemma4 MLX runs fine with the larger model needing 19GB.

Prior to that I had OpenClaw (in Parallels VM) create an application with local models that worked the exact same way as created by Claude. It was more an experiment in how OpenClaw works, hence the VM.

> And 57 gb is split across 3 cards quite easily,

I assume you are talking about a good graphics card. A good 32GB will run you $2K a card, so that $6K to beat out a laptop of similar price, and where the difference doesn't matter.

[edit]

Anyway my main point is Macs work fine for local models. I have a 6 year old machine that proves that.

Running agentic loops doesnt mean just being able to execute them. When your llm is so slow that you are faster writing the code yourself with free gemini that comes with google account, local llm is no longer worth it. Tok/sec is everything.

And 3090 is $1500 used, and 24gb gb of ram. 3x is $4500 for 72gb of ram.

> When your llm is so slow that you are faster writing the code yourself

I have never had that issue, and LLMs are used for more than just code generation.

> And 3090 is $1500 used

Second hand Macs are also cheaper.

Not sure what your point is, but you just seem to arguing because of your hatred of Macs.

The $8k machine will out preform the 1k gaming machine sure but it wont out preform anything in a similar price range.

Mine cost $5K probably because I only have ever needed 2TB of SSD.

But this is an old argument that seems to be thrown at Macs. Normally everything is cherry picked to show it's better than the Mac while ignoring all the features.

Although I would hope if you pay $8K you can get something that outperforms the Mac in every way.

Name one feature that Mac has that you think is so great.

It's funny because I was the same as you. I hated Macs with a passion. I saw them as overpriced shiny things that weren't as good as a PC.

Back in early 2000's I was commissioned to build an application that should also run on the Mac.

I bought a second hand old Mac Mini. Within a couple of months I bought a brand new one, and since then I have been on Mac when it comes to non-gaming (even though it can do gaming).

The main reason was I could just get shit done. The operating system is transparent to what I needed to do. With Windows you were forever dealing with the operating system to solve things.

Windows eventually got better in this regards, and more recently worse again.

I own/use windows as well. But mainly for gaming or a VM on an internal server.

Often when people give a comparison of how much PC is better, they compare a PC desktop to a laptop. Windows laptops for the same features is same or more expensive, while feeling cheap/heavy/noisy.

So form factor is one part of it, and then how the whole ecosystem works together with my phone/iPad and other devices.

At the end of the day, if I am doing any serious inferencing then I use my companies server, or Claude.

...

What has amazed me over the years is the sheer hate I get from some people for even owning a Mac. I was probably the same, but I was younger then.

Its died down since ARM came in, but the irrational hate seems wasted. Someone prefers Linux, Android, Windows, etc. Good for them.

Reply on news.ycombinator.com