Hacker News Reader

Comment by ajtulloch | Hacker News Reader

"It wraps cuDNN, which is faster than Tensorflow."

TF uses CuDNN, so this is a nonsensical statement.

No, it's not. Why does TensorFlow have separate benchmarks for Soumith's convnet comparison that show it to be significantly slower?

https://github.com/soumith/convnet-benchmarks

It says right there: "Some of the frameworks like TensorFlow and Chainer are benchmarked with CuDNN, but it is not explicitly mentioned"

So yeah, I'm also not sure what you meant by your statement.

But cuDNN itself is faster than those frameworks, so they are doing something to slow themselves down, and my statement stands.

Again, this does not make sense. CuDNN is the highly optimized code to perform a specific numerical calculation (e.g. convolution) on Nvidia GPUs. Frameworks such as TensorFlow or Deeplearning4j can use CuDNN to speed up its convnet calculations, but they don't have to.

In summary, your statement is like saying: "Numpy is faster than Python".

It feels strange having to explain these things to a creator of a NN framework.

That's a false analogy. What I'm saying is not like "Numpy is faster than Python"; it's like "Numpy is faster than some scientific computing that the creators of a Python framework hand-rolled separately."

Some frameworks write their own CUDA kernels for operations beyond what cuDNN offers, and also wrap cuDNN. So certain operations may run faster or slower depending on how you choose to execute them with the framework.

Ok, now we can agree.

Reply on news.ycombinator.com