Hacker News Reader

Comment by melling | Hacker News Reader

Any recommendations on learning PyTorch. I see many people on Kaggle who use it.

We did a bunch of popular research paper implementations in PyTorch with notes (annotations); might be helpful.

https://nn.labml.ai

@vpj The only drawback I see is that much of the implementation is abstracted by your helper libraries. Not everyone wants to add an extra dependency layer. Otherwise the walkthroughs are super helpful.

Agreed. But it makes implementing and testing a lot easier for us. We try to minimize the dependencies as much as possible.

Do you think it also comes in the way of understanding the paper and learning how to implement it?

Since the backbone of the implementation is packed away in the library import, I felt it didn't quite show the code & variable interaction well enough.

Don't get me wrong. It is useful & concise like you mentioned. But your target audience is beginners & adopters, and it makes it no different from another framework such as Fastai (I have major gripes with them. It has a much bottled-in experience)

To be true to walkthroughs, please consider designing helper functions rather than using your framework. Admittedly it may not be as beautiful, but eventually your users will be more appreciative of the extra mile you go into making things transparent & similar to PyTorch docs.

For people planning to learn PyTorch or even NN as a whole, it would also help to order or group the implementations in terms of complexity.

Your notebooks are very useful, thank you! May I suggest making their background white, or the color text less saturated? The keywords, function names etc. are very difficult to read, I have to paste the code in another editor.

There are lot of tutorials on the PyTorch website where you can start playing around with several cool examples in many different categories.

https://pytorch.org/tutorials/beginner/pytorch_with_examples...

Really depends on where you are coming from.

If you are already working in Deep Learning, and super comfortable in some other framework, then-

-> Head over to the PyTorch website, and go through the introductory tutorials.

-> Go to Papers with Code [0], and start reading and trying out implementing relatively easier research papers.

If you are a beginner, then, you can go through resources that reach Deep Learning through PyTorch. I recommend-

1. https://d2l.ai

2. https://atcold.github.io/pytorch-Deep-Learning/

Although, for DL professionals wanting to learn PyTorch will get less bang for the buck from these resources.

I recommend the PyTorch website tutorials for everyone, though.

[0]: https://paperswithcode.com

Fast.ai is basically pytorch + loads of utility functions. By following the course/book, one to some extend learns what fastai does, and how it uses pytorch for practical stuff.

I have also had good success with FastAI as my initial hook into PyTorch.

A brief post that shows just how thin the FastAI layer can be (if you want!) is here: https://muellerzr.github.io/fastblog/2021/02/14/Pytorchtofas...

I second this but once you are comfortable with fast.ai it's a good idea to reproduce some things in pure PyTorch eventually because it'll give you a deep understanding and appreciation for fast.ai as well.

One thing with fastai that annoyed me when I built a project with it was that the v1 and v2 APIs are totally different, and if you google or search stackoverflow for help with something, I found it more likely to stumble on answers for the v1 API than the v2 API. I also didn’t find their documentation super helpful for more than the most basic things (though not all documentation can be as amazing as scikit-learn).

Am I allowed to have a contrarian opinion :)

I went through the first version of FastAI (when it was Keras, torch?/tensorflow?) and forgot most of it because never did anything with Deep Learning. Then I did the FastAI V1 course again where they use FastAI library V2.

I really liked the first version of the course because I felt like Jeremy did an awesome job of balancing understanding the guts of using a DL library with getting stuff done. It was a tough course, but I felt like I really understood things.

I felt like the version with FastAI library V2 went too far into "Here are some commands you can use in the FastAI V2 library to to do this sexy thing with Deep Learning." I completed that course and really felt it should have been titled "A Course on the Fast AI V2 Library"

I recently purchased "Deep Learning with PyTorch" by Eli Stevens. I've been working through this book and feel like it explains things a lot more. I'm haven't finished the book with it, but I do like it so far.

I felt similarly about the fastai course. I'm doing the deeplearning.ai course (Andrew Ng) and I really like it. Warning though, it is bottom up, so you start by implementing neural nets with numpy to understand the internals, then it gets into TensorFlow. Just another resource if people are looking to learn deep learning and not pytorch in particular.

Reply on news.ycombinator.com