> If you can process "offline" for an hour and then cache the results, CPU inference is fine.
> GPUs are expensive.
Depends on the GPU. I've found T4 GPUs to be cheaper than CPU compute on AWS when testing throughput per $ of spend.
> GPUs are expensive.
Depends on the GPU. I've found T4 GPUs to be cheaper than CPU compute on AWS when testing throughput per $ of spend.
No comments yet.