Very true. This is the approach I've been taking. My project isn't human-interactive and I'm fine with it taking a couple hours for each run to finish.

The downside is that it makes development and testing super slow without the speedup you get from having local GPU power.

> If you can process "offline" for an hour and then cache the results, CPU inference is fine.

> GPUs are expensive.

Depends on the GPU. I've found T4 GPUs to be cheaper than CPU compute on AWS when testing throughput per $ of spend.