Since you mention it: Cloud TPUv3 is now available in beta (brand new!) See: https://cloud.google.com/tpu/

(420 TFlop/s, 128GB HBM)