The AI Boom Is Giving Rise to "GPU-as-a-Service"

I've been using a couple of these. The business model absolutely makes sense - low risk, let someone else maintain the hardware. You can just tear it down if and when your incredible "let's do it with an LLM or something" plan fails.

Additionally, if the provider knows what they're doing, they can put together very sensible specs and set non-stupid pricing.

On the other hand, the monthly expenses can quickly add up, to the extent that it becomes silly to not buy superior hardware for a fraction of say, the yearly cost of using such a service. Of course, that's probably a bad idea unless you have something valuable going on in development or in production.

"GPU-as-a-Service" absolutely does make sense as a business model. For instance, I have a pretty powerful GPU in my flat which I do all sorts of experiments on, but I'm not happy having a ~300W eGPU running in my residence 24/7.

Semi-related:

I thought this might be why Apple put the power button on the M4 Mac Mini in that stupid position on the bottom of the device. If they can keep more of them on then they can potentially grab them when needed to do some distributed work.