The catch is that the token generation speed is going to be limited by network latency, making it unbearably slow to run over the internet.

It can be great on a local network though, especially if your workload is prefill-heavy (more text input to process than output tokens to emit).