Anybody else has the feeling that PyTorch is to TensorFlow what Chrome was to other browsers? I started PyTorch about a month ago and was impressed how effortless everything was compared to TF.
Briefly, the benefits of PyTorch are
* easy conversion to NumPy arrays (meaning rest of Python can be used!). This is a bottleneck in Tensorflow; for reasonable sizes, PyTorch is 1000x faster.
* trackbacks are easy to follow (because defines graph by running)
* it’s as fast as tensorflow [2] (or at least torch is, which calls the same C functions as PyTorch, and there’s a tweet [4] by a core dev saying to expect the same speeds). Plus on the web I’ve only found anecdotes that support PyTorch faster than tensorflow.
* it’s easy to extend; everything is a simple Python class. e.g., see their different optimizers [3]
[1]:http://stsievert.com/blog/2017/09/07/pytorch/
[2]:https://github.com/soumith/convnet-benchmarks
[3]:https://github.com/pytorch/pytorch/tree/master/torch/optim
[4]:https://twitter.com/soumithchintala/status/83545486710789734...
PyTorch's imperative semantics (where the computation graph is implicitly defined at runtime by the execution of your Python code) definitely make it cleaner to do research prototyping. But AFAIK most PyTorch models need be reimplemented in lower-level code, or maybe something like Caffe2, before they can be used in production. That's a fairly significant tradeoff, which makes it hard to see PyTorch totally replacing Tensorflow anytime in the near future. That said PyTorch is obviously a great tool and it's exciting to see how it will develop and be used.
(disclaimer: I work for Google, opinions are my own)
No you can't because Google doesn't release those functions in the open source version of TF
Also PyTorch will soon by able to export directly to Caffe2 / CNTK
http://www.fast.ai/2017/09/08/introducing-pytorch-for-fastai...
Basically, some things were really hard in TF and in PyTorch they're easy. Time to insight while testing a new model is shorter.
I have not yet committed to a deep learning framework, as up until now, I was mostly either using scikit-learn or building neural networks from scratch, straight numpy (lol)
I've heard a nice thing about Keras is that it forms more of an abstraction on top of other libraries, though I could be misunderstanding.
I also tried to flip the keras switch to run on top of theano instead but it had issues that I didn't have time to fix, so I just stuck with the original lasagne/theano stack.