Comment by _coveredInBees | Hacker News Reader

Comment by _coveredInBees | Hacker News Reader

We use Pytorch extensively in our startup. We tackle a lot of new research problems as consultants/partners to help develop products or devise new algorithms/models to solve tasks for our customers. We have never regretted our choice to pick Pytorch. I found the article pretty spot on when comparing Tensorflow and Pytorch. The things that have appealed to me about Pytorch are:

1. Extremely easy to debug and work with. Being able to debug effortlessly in PyCharm makes life very easy.

2. The API is quite clean and nice and fits in really well with Python and nothing feels hacky. I've developed my own Keras-like framework for experimentation, training and evaluating models quickly and easily and the entire experience has been really enjoyable.

3. The nicest thing though is that as the article points out, a huge percentage of researchers have moved to Pytorch and this allows us to more easily look at other researcher's code and experiment with things easily and incorporate ideas and cutting-edge research into our own work. Even for things that are released in TensorFlow, if it is an important publication that gains attention and traction in the community, you will likely have implementations in Pytorch pop up soon enough.

I do think that TensorFlow still has an edge on the deployment at scale/mobile side of things as pointed out by the article. But Pytorch is a lot younger and they are making a lot of progress with every release in that space.

Last year I was tasked with looking into a NAS (Neural Architectures Search) paper and analyzing the algorithm. The paper came with a TensorFLow implementation. Trying to read that TF code was quite difficult. I searched around and found a PyTorch implementation - much easier to read and understand, and it ran about 50% faster as well (the latter was a bit surprising). I tend to think that TensorFlow lends itself to the creation of code that's difficult to reason about. That may be different now with the various flavors of TF (like TF Eager).

I'll add that it was much easier to install PyTorch with GPU support than it was to install TensorFlow with GPU support - at least that's how it was around November of last year. The PyTorch install was painless, whereas we ended up having to build TF from source to work with out setup. Could be different now as I haven't looked at TF since then.

Yes, installing Pytorch with Cuda support has been unbelievably simple. I've never had trouble with simply pip installing it in my venv.

> That may be different now with the various flavors of TF (like TF Eager)

Unfortunately, if anything I think it's the opposite. The constant creation and deprecation of TF flavors (tf-eager, tf-slim, tf-learn, keras, tf-estimator, tf.contrib [RIP]) has made reading tensorflow code online somewhat disastrous. Everybody, including the TF team, is using a different API and it's difficult to keep all of them straight. It seems that you're doomed to end up using some combination of many of the above in a way that makes sense to you and your team, adding another confusing model to the pile.

Agree overall, but tf.eager doesn't have much to do with the rest of the list.

tf.contrib is just a module where user-contributed code was stored, which included both low-level constructs and higher level APIs. tf.estimator is an abstraction that is mostly used for productionizing models. tf.slim/tf.learn were indeed redundant with keras (a library developped externally), but were necessary steps before keras became part of tensorflow.

I will add further to this point.

There will be bad model code in both PyTorch and TensorFlow. The difference is that, bad PyTorch code reads like bad Python code, and I've accumulated a lot of experience reasoning through bad Python code. Bad TensorFlow code can come from any one of of the history of paradigms that TF has gone through, and I don't even know if it's bad or just some funky new TF functionality I'm unfamiliar with.

> I don't even know if it's bad or just some funky new TF functionality I'm unfamiliar with.

Yes, Javascript features and frameworks have evolved quite fas... oh wait

What made me fall in love with PyTorch was also that the "neural network training process" is defined almost as it is in theory, in code in PyTorch

- loop through epochs

- loop through each batch

- run a forward pass for the batch ( model(batch) )

- calculate the loss for the batch ( criteria(y, yprim)

- compute the gradients/backprop ( loss.backward() )

- update the weights (optimizer.step())

This really enforced everything I learned and I think breaks down the problem. All this of course in addition to everything else already mentioned, and super convenient module/network building and definition.

Yes! Exactly.

Agreed, but projects like Cortex are narrowing the gap around deployment imo https://github.com/cortexlabs/cortex

Seems like an AWS specific thing?

For a moment there I thought you might be referring to https://github.com/thinktopic/cortex

What does your start up do?

Reply on news.ycombinator.com