Genuinely, everything starts feeling super sluggish, even long after the process has terminated -- cmd+tab gets slow, apps constantly hang for a few seconds at a time, the cursor position starts lagging and videos are slow to pause/play -- and the only way I've found to fix it is to reboot the entire machine. Clearing memory, restarting apps etc. doesn't work.
Don't get me wrong, the performance and QoS system works really well while the model is still in memory, but after a few cycles of loading and unloading, performance just ends up getting bad in general. Doesn't matter if I'm loading it with llama.cpp, MLX, native Metal, if I start using the GPU for machine learning the machine just gets really upset.
This sucks man... macOS is not supposed to need reboots like that, my Intel Mac could run for months on end without slowing down at all and I could push it as hard as I wanted. As far as I can tell, this behavior is new as of either macOS 26 or the 27 beta. Apple really has been betraying their true Unix roots lately.