Hacker News Reader

Comment by prettyblocks | Hacker News Reader

My problem is how hot they run. I'm on an m4 pro. Do you have the same issue?

It’s hot and also LOUD and runs the battery down quick.

But I’m having a lot of luck just running things when I’m away from the computer and can leave it plugged in.

It starts going weird (unreliable and slow) with context over 80k so you have to pick tasks one at a time and baby sit a lot more than Claude. But it really is very capable and feels like there’s an intelligence there to talk to. Maybe gpt-4 level clever?

I have an m5 max 64gb and I think anything slower would be quite painful.

>It’s hot and also LOUD and runs the battery down quick.

Who'd run this on battery?

Or do you mean kills the battery faster even when used while charging?

There was a lovely window of a few years when processors were fast enough and low-power enough that real development work could trivially happen on a Macbook Air in a lounge.

That window is waning as more and more memory and graphics processing power is being used locally.

I suppose the future is most likely going to involve farming out AI requests to your desktop machine, your company's compute farm, or a cloud endpoint, but if you're expecting to run an AI offline on your lap with a battery...your lap will get warm.

Running your dev environment on your laptop is so 2025.

"So 20XX" is so 2012.

I envision a future when the potential energy in a laptop battery, could take out 17 city blocks if it exploes.

No idea what you are talking about. My battery lasts longer than ever while running vim and make and GCC. It’s amazing. Not sure why your windows are closing.

Because the local LLM, which you are not running, is running for much longer than gcc and is eating the battery. Different choices, different outcomes.

Bur the implication here in the parent comment is that for ‘real dev work’ to be done you must be running an LLM, and therefore you can’t do it on a MacBook anymore.

Turns out you can still accomplish stuff with a text editor and compiler.

Of course we can code without a LLM, but if we want to, as the sibling comment says, don't run a local LLM on a laptop. I'm offloading the LLM to the LLM vendor, like most of us IMO. The CPU is still busier then it used to be, because of the local harness.

> There was a lovely window of a few years when processors were fast enough and low-power enough that real development work could trivially happen on a Macbook Air in a lounge.

I was responding to this. I am appalled that anyone thinks (and is willing to say out loud in public) that they cannot do "real dev work" without an LLM.

So don’t run a local LLM?

Running the laptop hot ages the battery even when you’re plugged in.

Merely existing noticeably ages most battery chemistries.

You can take half an hour out of every week inconveniencing yourself to protect your battery, or you can spend an hour once a year just putting in a new one (or having it done for you in-store if you're using an Apple device) and save yourself the nagging worry.

I was trying to do some work on the train with no signal.

Couldn't you just enjoy the ride?

I have an m3 max 64GB and it runs ok, I was playing around with it last night. Its reasoning is better, but the token/s still makes an MoE model (35B A3B) preferable so I switched back.

Try using Goose instead of Claude's harness? Goose doesn't load as much context in at the start, so it might be more usable. I've definitely been able to get it past 128K, although I typically don't go beyond 70-80k. MoE might also be better at this.

If you're running it while idle and don't need the quickest results, reducing the clock speed improves energy efficiency (and in your case avoids overheating the battery). There will be some optimal speed that maximizes computations per joule that depends on the specific load and can only be found by measurement.

On Linux, you can cap CPU frequencies with "cpupower". Does MacOS have any equivalent?

The Mac unfortunately just has two performance modes ‘all out power and melting’ or ‘cold and really really slow’

My M5 Max has 3 modes, "low", "automatic", and "high". Automatic doesn't seem to simply switch between low and high -- it seems to sit in the middle and vary dynamically based on workload and temperature. With automatic I get more than half as many tokens per second as on high, but with much lower fan speed and temperature.

I remember doing this with crypto mining way back in the day. That machine will die within a year.

Macs fan control set to full blast and lifting the Mac above table for air circulation across bottom.

I use M1 Max with qwen3.8 27b mlx. The gpu temp can reach easily to 95°C as fan doesn’t kick in automatically until 90°C. With Macs fan control at full blast and MBP off table, temp usually hover around 85°C.

Yikes isn’t 95 well above the danger zone for most components or have things changed? I remember crypto mining in the early 2010’s and making sure to stay under 85C on my GPU.

Stuff runs so hot these days. I think one of the parameters pushed to keep Moores law from completely flattening was heat. My desktop regularly hits 95 just compiling code.

I guess thermal throttling will inevitably kick in.

Just curious, what is your ram and how many tokens per second do you get with that?

RAM 32GB, tokens are less than 20, may be about 12-15. Main issue is context window, I have set to 64k but practically beyond 32k, too much swap and slow down.

I am getting like ~25 tps with M5 Max 64GB. Unsloth Desktop and Qwen 3.8 27B Q6 KXL

If you're going to try to use a mac laptop for long term inference you'll probably want to buy one of those $30 laptop fan cooling pads (sold for absurd sized 17 inch x86-64 gaming laptops) on Amazon. Otherwise you'll roast the battery over time and significantly hurt its cycle life and capacity. If you can get significant airflow under the laptop that'll help a lot.

I don't have the hardware but a often mentioned advice is to put your mac into energy saving mode - it still will work, a bit slower, but stays cool.

Imagine spending all that money on Apple hardware only to throttle it to a fraction of its performance lol

Steve Jobs would be proud. People really are holding their Apple hardware wrong.

I tested Gemma 4 a couple months ago and yes it ramps the fans to max. Definitely a thing that should only be done on wall power.

Having a server in the basement helps a lot :-D Then tailscale from everywhere.

Mineral oil bath?

just decent air cooling and you'll be okay. it will get up to 75/80C though for my M5 MBP.

Reply on news.ycombinator.com