My back of the napkin calculation says that I can get 270 petaflops with 2160 n1-highmem-16 each with 8 v100 gpu on preemptible instances costing roughly $13k/hr or about $10m/mo
AWS looks like a bad deal here.
If you have a variable load, cloud infrastructure may make sense if you can easily auto-scale.
In my experience, most business real world applications are multi tiered applications with variable loads hence are a good fit for cloud infrastructure.
However, attaining the required application flexibility and KPIs for efficient auto scaling is quite hard and require strong functional & technical expertise.
I'm running infrastructure for a SaaS app in k8s. I feel like I'm doing well sustaining >50% efficiency, i.e. all cores running >50% all the time and more than half the memory consumed for things that aren't page cache. Hard to get better efficiency without creating hot spots.
Not a great deal.