> It is much more likely for your model to fit in large unified memory of a Mac than the smaller more limited memory of a GPU.

I bristle at general statements like this when it obviously depends on the specific Mac and GPU in question. But yes, comparing a maxed out M5 Ultra with an RTX 6000, the Mac has much more memory.

> Even going with two 5090s, you now have to shard your model and that is a PITA.

Every modern tool does this for you automatically. It is absolutely not a pain in the least (e.g. llama.cpp ships with pipeline parallelism enabled by default).