Mystery: Why do some LLMs produce more coil noise on Mac Studio M3 Ultra?

twitter.com

9 points·behnamoh··6 comments

I just gotta say, the coil noise on my open-case 3090 has been a fascinating and joyful complement to my generative experiments.

Alongside `nvtop`, coil noise has given me a visceral sense of the texture for different workloads. Sorry I haven't done a more rigorous analysis, but I'm certain it helps me debug.

In video games this tends to be a function of framerate, with higher FPS being more likely to produce coil whine as the GPU power draw oscillates at a higher frequency. I assume there's something analogous in LLM runtimes when the outer loop spins faster or slower.

in this case, the smaller glm mxfp4 model actually runs slower than the much larger m2.1 model.

that tracks, it's not staying loaded and is buzzsawing back and forth between power states. They'll likely fix it with some power management tweaks. Had a pair of modded Vega 64s that sounded like a tiny CNC mill when they did this, but that's what ~900W will do if you let it.

I haven't used a mac in a while but there's probably some values you can fuck with if you disable sip to stop it from ramping back down so aggressively, or keep it in the higher power states.

Back in the early days of deep learning I did a fair amount of my performance debugging by listening to the coil whine during training, (this was when a GTX 1080 was hot stuff.) Fun to see this come back!

My amd ryzen 9 9800x3d with ASUS b850 motherboard whines like a mofo. I used to run ai tooling front ends that launched workloads into the ryzen9 backend. Randomly I would hear it whining and could guess the workload based on the whine.

Reminds me of my Intel 486dx2 66mhx systems or pentium 200mhz cpus which would generate cool whine on 97.1 through 100.3 on my radio in my room.

Ahh nostalgia

Reply on news.ycombinator.com