Hacker News Reader

Comment by solomatov | Hacker News Reader

I wish Google created a development version of TPU for inference so that it's possible to debug models locally and then send them for training to the GCP.

View on HN

Ya, I feel uneasy about this business model of creating hardware that you have to connect to a cloud service to actually use. Instead of vendor lock-in or proprietary drivers or whatever, it's a new form of locality-based lock-in.

Meaning, if I have an application that needs a big hot PCI-E card attached to a physical server I own somewhere, comparable to GPUs now, the TPU is not for me. But meanwhile, a bunch of NN research and frameworks on top of TensorFlow will treat these proprietary things as a first class citizen.

The lock-in, while bad, only affects the development of new models. These devices exist so that you can avoid the cloud for inference.

Well, you could get one of the development boards? Not sure what the use case you have in mind is, but these Edge TPUs are not for use inside of their GCP AI solutions, but rather on an 'edge' device. For anything else your 'local TPU' should be a hefty desktop with a couple of NVidia cards.

Google Colab provides free TPU/GPU accelerated runtimes https://colab.research.google.com/

TPU in Colab is not the same device as edge TPU.

The USB accelerator is designed for that - so you can target models for the a (small scale) TPU and then scale up on the cloud.

This would specifically let you make sure that the TensorFlow ops your algorithms use are supported on a TPU.

https://coral.withgoogle.com/products/accelerator/

USB has too little bandwidth to do real training. Modern GPUs use 16PCIe lanes which is ~126Gpbs for PCIe 3.0 which is incomparable to USB.

Hence the part "scale up on the cloud" - USB units aren't supposed to replace the GPUs, c'mon.

The ask here was inference, not training.

You could buy your own, but I don't know how different the Edge TPU version is.

http://linuxgizmos.com/google-launches-i-mx8m-dev-board-with...

It's very different. It's optimized for inference not for training.

An emulator?

It's too slow for this.

Is it?

The edge TPU can do MobileNet V2 at 100 FPS.

An iPhone 7 can do it at 145 FPS (source https://machinethink.net/blog/mobilenet-v2/)

The deal is though that the Edge TPU is able to do it at much lower power.

Reply on news.ycombinator.com