Well, the actual inference providers put each expert on its own single GPU. Deepseek explicitly does this.

Read-only parameters is also usually the majority of space. Deepseek is 700GB of params. Meanwhile kv cache is small (Deepseek is about 7GB at max context) and ssm/conv1d cache is even smaller- IIRC Qwen 3.5 is 146MB per token regardless of context size. Not sure about how Mamba-3 works, but I suspect read-only parameters are still a significant amount of memory bandwidth.

I guess the question isn't whether compute is 1:1 with memory, but rather if you run out of compute before you run out of vram adding more users.