At present, preemptible Cloud TPU v2 and v3 devices are widely available, and they are likely to be the most cost-effective option for training any of the models listed here, often by a wide margin: https://cloud.google.com/tpu/docs/tutorials

Definitely appreciate your point about comparing Summit to your desktop - however, the difference is that you can't rent Summit via any public cloud, whereas you _can_ rent Cloud TPU Pods.

You might find the comparison between 8 x V100 GPUs on GCP and a full Cloud TPU Pod more relevant - in that case, as of the time the Google Cloud blog post linked above was published, a full Cloud TPU Pod delivered a 27X speedup at 38% lower cost for a large-scale ResNet-50 training run, all without requiring any code changes to scale beyond a single device.