Is there any public access to the TPUs?
For models you write yourself and have the Cloud ML service train, they will use our regular VMs today and as this blog post says, GCE will soon have GPUs (and therefore, Cloud ML will expose them, too).
Disclsoure: I work on Google Cloud (and sorry for the confusion!)