Hacker News Reader

Comment by londons_explore | Hacker News Reader

It would be good to see it compared to a TPU...

I feel like thats becoming tensorflows 'native platform'...

TPUv1 were made to have good performance per watt for inference. [1]

TPUv2 were benchmarked against NVIDIa's K80 at 30 times the performance and had a peak of 92 TOPS [2] while the 2080 TI is at 440 TOPS [3] [1] https://www.anandtech.com/show/11749/hot-chips-google-tpu-pe... [2] https://www.tomshardware.com/news/tpu-v2-google-machine-lear... [3] https://www.microway.com/knowledge-center-articles/compariso...

Why would anyone compare a modern tpu against an Nvidia card that is four generations old? The tpu didn't even exist when the K80 came out.

Marketing.

nVidia is destroying the TPUs right now and Google is desperate to keep their perception in the public eye as the king of AI (which tbf, they probably are, compute capabilities aside)

You should try the assumption of good faith some time. It generally leads to a happier place.

In this case the data was from the time Google introduced the TPU internally, when the K80 was very much up-to-date. It also makes sense because the K80 was the only GPU offered in GCP.

This is the tpu v2, and it was most certainly not released internally when the K80 was.

You're talking about the tpu v1, not v2 or v3, so no.

Also, there's no need for assumptions when you know what's going on in the design team.

Google usually talks about it's hardware few months after they have moved to a newer better version. They probably are already using TPUv4 by now.

TPUv1 was never exposed externally as a thing you could rent in cloud, but with that exception, Google's been pretty consistently trying to offer the best available option to cloud customers.

(disclaimer: while I'm part-time at Google, this is my personal impression, not an official statement, etc., etc.)

They probably feel TPUv3 is stable enough to give access to cloud customers now. But I doubt they do not already have TPUv4 versions running already internally in some capacity maybe only for testing.

Since you mention it: Cloud TPUv3 is now available in beta (brand new!) See: https://cloud.google.com/tpu/

(420 TFlop/s, 128GB HBM)

4x TPUv2 is roughly equal to 4x V100 performance wise.

The price/performance ratio of rented TPUv2 or V100 can't match the price/performance ratio of owning the system if you are doing lots of learning/inference.

If the model fits inside 2080 Ti and the work is not tightly time restricted, 2080 Ti (the whole $2.5k system) should be more economic choice after six months or less (full utilization 24/7).

Really can't do a directly comparison but we can look at cost doing a similar task.

DAWNBench does benchmarks.

"At the time DAWNBench contest closed on April 2018, the lowest training cost by non-TPU processors was $72.40 (for training ResNet-50 at 93% accuracy with ImageNet using spot instance). With Cloud TPU v2 pre-emptible pricing, you can finish the same training at $12.87. It's less than 1/5th of non-TPU cost. "

Here is a link to DAWNBench.

https://dawn.cs.stanford.edu/benchmark/

Reply on news.ycombinator.com