Hacker News Reader

Comment by maldeh | Hacker News Reader

Another potential source of friction I've wondered about regarding getting research workloads running on TPU2 is the heavy reliance of most of the popular DL frameworks on CUDA and CuDNN; I don't understand how one could port those libraries to a drastically different architecture such as this. Some DL frameworks have made overtures to OpenCL but it has not typically been a top priority [0].

Could this instead imply Google is working on its own library of low-level math primitives just for Tensorflow, and if so, how long would that take before it's competitive performance-wise? At any rate, having to support a different computing platform / API would be another blocker to general adoption.

[0] https://github.com/tensorflow/tensorflow/issues/22

View on HN

Would this not be solved by TensorFlow XLA?

https://www.tensorflow.org/performance/xla/

Per my understanding of XLA, it provides the right high-level abstraction for compiling the tensorflow computation graph under a range of architectures, CPU, NVIDIA GPU, mobile, etc. but it still delegates to a lower level domain-specific API like CUDA.

This undertaking isn't something to be taken lightly; CuBLAS has some pretty cutting edge architecture-specific optimizations for batching operations for matrix multiplication that came from several years of research - and is arguably a massive competitive advantage of NVIDIA over AMD. Depending on the development state of such an API internally within Google, it could mean that the Cloud TPU isn't going to be ready for wide-spread commercial use for a good while, and is very much still in the research-and-development phase (which could explain why they're only opening it up for the research community right now).

From what I gather from reading AMD's recent AMA is that the key is getting hardware and software guys [working together](https://lists.freedesktop.org/archives/dri-devel/2016-Decemb...). For AMD the issue is that they can't afford to have hardware guys dedicated to helping optimize the software since they need to build the next and next next generation of hardware. On the other hand, Google surely has the pockets to get man hours and knowledge to (a) figure out what kind of software is needed (b) design from hardware level to meet their software needs.

And if you caught any of their talks on [Tensorflow at Google IO](https://www.youtube.com/watch?v=5DknTFbcGVM), it seems like their goal is to provide ultra-high level APIs like POST/GET, and high level python ones like Tensorflow pre-built and trained models. At that point Google can control the engineering, and as long as you run on their cloud platform you just have to worry about the high-level. I'm definitely not a fan of vendor lock-in, but it seems like a interesting product, and I'm curious to see what Google does in this space.

You could specify computation in high level API like TensorFlow, and then have the framework pick the best implementations available (ie, CuDNN for GPU, MKL for CPU, something custom for TPU)

"something custom for TPU" is what seems interesting here. Researchers working on the TF research cloud probably wouldn't learn much about the sorts of workloads this architecture is suited to, if all they have access to is the high-level API. Would whatever iteration of the TPU API they have internally be something they'd release soon, at least to partners?

Reply on news.ycombinator.com