Introduction to Recurrent Neural Networks in Pytorch | Hacker News Reader

Introduction to Recurrent Neural Networks in Pytorch

cpuheater.com

184 pointscpuheater17 comments

Sound like a pretty neat introduction! This is exactly the kind of thing I needed, coming from tf/keras and looking to switch to pytorch for research

As someone learning Keras right now, why are you wanting too switch?

Not the parent, but the imperative interface supported by the dynamic graph approach Pytorch takes is much nicer.

Additionally, in my personal opinion Tensorflow is often too low level and Keras is often too high level for the things I'm trying to do for research. While you can jump between the two of course, I think PyTorch hits a much more natural middle ground in its API.

Tensorflow/Keras is making improvements in these areas with the eager execution, and is still great for putting models into production, but I think PyTorch is much better for doing research or toying with new concepts.

This article has some good comparison: http://www.goldsborough.me/ml/ai/python/2018/02/04/20-17-20-...

Also, I believe the newer fast.ai course uses Pytorch, and the prior used Keras?

I would like to know more from the article about setting x,y,m1, and m2. Any explanation is appreciated.

any reason why you wouldn't use tf eager?

tf.eager is much slower than pytorch.

this is a strong claim. what is your source for this? I'm legitimately curious.

No hard benchmarks, just personal experience. Note that I’m not saying regular Tensorflow is slower than pytorch (in fact I’ve found them to be roughly the same) just eager mode.

Edit: Just realized this might be a good thing to write a blog post about. I’ll get back to you after finals :)

I'm genuinely interested if you ever do write that blog post

Update: other people have already scooped me:

https://medium.com/@yaroslavvb/tensorflow-meets-pytorch-with...

I've seen similar performance regressions on my own tasks and I don't have much to add beyond what's in that blog post.

I assume this is more intro to Pytorch than intro to ML.

Any tips on an high quality intro to ML content using PyTorch for the hands on examples?

The Fast.ai courses are great.

Seconded. Their courses are superb, and they have their own library built on top of PyTorch that makes creating high quality models even easier.

You start with their lib, and over time they teach you all the techniques they're using, so the easy black box you start with becomes more transparent over time. It's a hands-on, code-first approach.

This is an interesting introduction to writing your own neural network models from scratch in PyTorch.

I don't think it's a great way to learn it though - almost no one writes their own models from scratch.

Almost all the time you want to be using one of the pre-written RNN models, since they are optimized, debugged and do things like use CuDNN where available.

I'm not sure what's the point. Predicting the sine-wave is pretty trivial with NN, and doesn't require a RNN.

It's an educational post. Calling something "pretty trivial" doesn't reduce the value of the post for people who don't know what you know, and want to learn it.

That wasn't my point. How can a post teach recurrent networks, if the "recurrent" part of it is redundant, and the network would work perfectly well without it?

Reply on news.ycombinator.com