Have been experimenting with deploying to Mac's for inference as they are cheaper and use less power. But also still deploy to a substantial amount of gpus too.

Either way I try to take connections to huggingface hub out of the equation at deployment time.