Good for training, definitely a bad idea for inference. But if you are spending that much money, why not just buy the equivalent of GPUs? You could buy 10 12GB 3060s for that price.
Powering ten 3060's and having a computer that can accept ten GPUs becomes a non-negligible hurdle to overcome.
For LLM developers, is there really no advantage to having a big block of unified memory, rather than a bunch of devices with a small amount of memory each?
MoE inference wouldn't be terrible. That being said, there's not a good MoE model in the 70-160B range as far as I'm aware.