How to Start Learning Deep Learning | Hacker News Reader

How to Start Learning Deep Learning

ofir.io

264 pointsofirpress29 comments

From the point of view of a programmer without much background in advanced maths, this is the best short intro to the subject I have found so far:

https://medium.com/@ageitgey/machine-learning-is-fun-80ea3ec...

Really enjoyed reading through that 3 part series. Easier to understand for an average programmer. Another one along those lines: https://www.oreilly.com/learning/hello-tensorflow

On a side note, one interesting thing about deep learning is that it requires less advanced math than other branches of machine learning. High school level or early college level should be enough.

I am very sorry for the negativity, but that post is just a list of links to Deep Learning and Machine Learning classes compiled by a student. The fact that it made it to the front page of HN proves that there's a massive hip about Neural Network among programmers, but that it sounds like black magic for most people interested in learning them.

Lots of people have heard about deep learning but dont really know where to start. When I was starting a year or so ago I was in the same place. I just took the resources that I used to learn about the field and put them all in one place. As far as I know, this hasnt been done before. Im not trying to provide any new insights into deep learning.

Greg Brockman (Founder of Open AI) has written this amazing answer:

If you want to read one main resource... the Goodfellow, Bengio, Courville book (available for free from http://www.deeplearningbook.org/) is an extremely comprehensive survey of the field. It contains essentially all the concepts and intuition needed for deep learning engineering (except reinforcement learning).

If you'd like to take courses... Pieter Abbeel and Wojciech Zaremba suggest the following course sequence:

- Linear Algebra — Stephen Boyd’s EE263 (Stanford) - Neural Networks for Machine Learning — Geoff Hinton (Coursera) - Neural Nets — Andrej Karpathy’s CS231N (Stanford) - Advanced Robotics (the MDP / optimal control lectures) — Pieter Abbeel’s CS287 (Berkeley) - Deep RL — John Schulman’s CS294-112 (Berkeley)

(Pieter also recommends the Cover & Thomas information theory and Nocedal & Wright nonlinear optimization books).

If you'd like to get your hands dirty... Ilya Sutskever recommends implementing simple MNIST classifiers, small convnets, reimplementing char-rnn, and then playing with a big convnet. Personally, I started out by picking Kaggle competitions (especially the "Knowledge" ones) and using those as a source of problems. Implementing agents for OpenAI Gym (or algorithms for the set of research problems we’ll be releasing soon) could also be a good starting place.

Quora link: https://www.quora.com/What-are-the-best-ways-to-pick-up-Deep...

Thats awesome, I wasn't aware of this. I slightly disagree with Greg though about the "one main resource". While the DL book is amazing, its not really aimed at newbies, and so I think CS231n is a much better starting point. For example, for a beginner I think the way backprop is explained in Stanford's course is better than the explanation in the book.

"One main resource" as in, it goes through all the underlying math required in details etc. It's usually assumed that people entering into DL have some experience with Machine Learning. Of course, for someone staring with ML, CS229 is the first thing she should pick up.

I just wanted to say this is a great recommendation for anyone who is serious about learning deep learning / machine learning. I also want to note, there are other resources available for people who are less serious about it and just want to get a general overview.

I would ignore the top reply. I found this a very helpful collection, and I appreciated your breaking down of resources for starting off in different areas of deep learning. It's at the perfect level for a summer student that I'm currently supervising.

I'm "on the other side". I took a very good AI course 6 years ago, right before deep learning exploded. And while I had the concepts to understand why they work, it took me a while to start using the new tools.

The compilation of links is good, and you are very clear presenting them. Wish I'd had this earlier to save some time, but still, thanks for compiling this list!

I found it reasonably helpful. Especially the line telling me what Maths prerequisites I should have and where to get them.

I found this list useful, and I appreciated the context and alternatives given to the standard recommendations. However, this comment epitomizes what drives me nuts about HN, negativity, with no feedback or justification.

What, if anything is wrong with the list of links provided?

What would you have recommended? And why does this list come up short?

What led you to believe the link would be more than "just" a list of learning classes?

You take issue with the fact this list was compiled by someone getting their master's in Deep Learning, why? If it matters who gives the suggestions, might I ask who you are and what qualifications you posses to so flippantly discount someone's contribution?

> proves that there's a massive hip about Neural Network among programmers

Wouldn't the opposite be true, if people learned more about the subject ?!?!

> I am very sorry for the negativity

Are you?

I think he's more upset with the fact that it doesn't quite live up to the title it holds, and is disappointed it's just a list of deep learning courses most people (who have bothered to look into learning about ml) know about already.

I also think you're taking this too personally.

If you ask me about ML I'll say that starting with convolutional nets for images is starting on the wrong side (end instead of beginning). Andrew Ng course is better, but still somewhat lacks theory and general ML insights.

I'd recommend new Coursera Washington University ML course (https://class.coursera.org/machlearning-001/lecture) as a solid foundation. You'll learn key ML concepts (bias vs variance, bayes theory etc.) and the lector is simply the best I've seen.

Following that Ian Goodfellow book http://www.deeplearningbook.org/ is very good.

If you want to learn foundations of Reinforcement Learning I'd recommend Sutton & Barto https://webdocs.cs.ualberta.ca/~sutton/book/ebook/the-book.h...

+1 to Sutton & Barto. There is a second edition of their book available; see a link at https://webdocs.cs.ualberta.ca/~sutton/book/the-book.html.

https://www.udacity.com/course/reinforcement-learning--ud600 is also good; it is not super-modern, but it is easily accessible, and it covers a large part of classical RL - this helps if you want to read recent RL papers because terminology and ideas become more familiar.

I'm not sure Andrey Karpaty's blog post (linked in the article) is a good intro to RL - it starts with Policy Gradient which is on a complex side of RL techniques spectrum. From other sources I've heard Policy Gradient is harder to get working on an arbitrary problem than e.g. Q-Learning, but don't quote me on that :)

If one doesn't want to dive deep in RL, and just want to start with recent methods then https://www.nervanasys.com/demystifying-deep-reinforcement-l... blog posts provide a very good starting point.

Shouldn't it just be a helpful list of links? "How to Start" == "Here are a bunch of links to resources I found helpful".

And like another commenter said: comments that are negative for the sake of being negative are why HN has a negative reputation.

On a related note, can someone give a glimpse into what is it like to actually change careers from development to ML? How much do you need to learn, what skill level is required, and how do entry ML positions compare with regular mid-senior developer positions in terms of compensation and overall work environment?

Yeah I'm interested in this as well, and cannot find the information anywhere...

I know of three paths right now:

- Get a Master's or PHD

- Work at Google and wait for your turn to do their internal ML bootcamp

- Udacity Nanodegree Plus (I would love to hear from anyone who has gotten a job through this)

I read the best way was(no actual experience in this field, but it sounded like a good plan to me)

1. Finish a degree in ML and a couple of projects

2. Use this experience, as evidence of enthusiasm for applying as a support engineer on an ML project. (Working on infrastructure, and all the surrounding development work an ML project requires)

3. Keep trying to move more and more into the ML side of things.

The article mentions https://www.coursera.org/learn/machine-learning

does anybody know:

- How much work is required by week?

- Pre-requisite?

- Programming language used?

Other feedbacks?

There's no prerequisites, Andrew manages to make all the maths self contained. He uses MATLAB/Octave, and the assignments can take around 8 hours or so of work.

I'm currently going through the course (at week 5 at the moment).

Work per week: can vary between 6 to 12 hours depending on the week and your background and willingness to dive into some demonstrations.

Pre-requisite: first, you definitely need to save some quality time otherwise you'll drop out (I heard the completion rate is around 10%). I feel that having a bit of matrix computations, algebra background helped. Some parts are easy, other parts are harder (but YMMV).

Programming language used: octave; while it's nice to work at that level of abstraction, I found the "feedback loop" very slow when you submit exercises, so in the end I used https://github.com/MOxUnit/MOxUnit with http://entrproject.org/ to have a faster feedback loop (anyone can email me at thibaut.barrere@gmail.com to get more details). This is especially useful when dealing with vectorization of computations.

Overall I found that the course is great for me (coming from an ETL, programming background with some solid maths exp at some point), and I'm learning quite a bit; a good introduction with a pragmatic viewpoint.

I just finished the course last week. I really liked Andrew's teaching style and enjoyed it very much. The work load is light compare to on campus courses at my school. I strongly recommend anyone interested in ML/AI to take the course.

I work in Medical Devices. In one of my jobs, the goal was to collect blood products - Platelets, Plasma, RBCs and keep WBCs at bay - using centrifugal separation. The 'prediction' algorithms would make a best effort prediction on how fast (and when) to run the various mechanical components - the Centrifuge, Platelet, Plasma and RBC pumps to optimally collect these products with minimum variability. Luckily they had the foresight and technical chops to build and collect every little detail about how the devices ran, under what inputs etc. Access to the 'outcome' of the treatment was tricky - the hospitals/collection centers had to actually measure the actual platelet counts and make sure they enter it into their system. What I'm getting at is that there is a trove of data in Medical Devices - if there isn't, hopefully companies will start collecting data with a view to applying machine learning a few years down the line - and given the complexity of some of the problems, this would be a great field for application.

A couple of years back, there was also talk of Pharmaceutical companies opening up their data for clinical trials. I'm not sure where things are with that effort. If anyone has any info on that, it would be good to know.

Here's some free machine learning books to get you started, as well - https://hackerlists.com/free-machine-learning-books/

Reply on news.ycombinator.com