Hacker News Reader

Hacker News

Comment by mrdrozdov | Hacker News Reader

Full thread

mrdrozdov

My top reasons why everyone getting into maths/stats/cs should go straight for deep learning:

a. recent findings are documented incredibly well in both research and code

b. because of its success, there are many areas for useful contribution at relatively less effort from the researcher

c. because of its success, it'll help you develop marketable skills

d. it's fun

Maybe it won't solve General AI, but it seems like a damn good foundation for the person/people that will eventually come out with ideas that move us closer in that direction.

View on HN

Which paper/blog post/book specifically documents the most recent research? E.g. just yesterday I figured out (after surfing the internet for a long time) that the diminishing gradient problem have essentially been solved by ReLUs, so auto-encoders are no longer necessary and we can train a whole deep network at once. Another is e.g. batch regularization. Probably not news for a researcher, but where could I learn these updates?

The only way to really stay caught up is to read the papers that are being released (NIPS, ICML, ICLR, CPVR, etc). If you use a course or guide, then you're behind by definition. If you're okay with being a little behind, then the Deep Learning courses for NLP or Computer Vision are both excellent. NYU also has similar courses. You can also try following some major Deep Learning people on Facebook/Twitter as they post about the new stuff daily. People to follow: Lecun, Lawrence, Bengio, Cho, Karpathy, just to name a few. Interested to hear others ideas on how to stay up to date.

I disagree that a. is true. When I have read DL papers I have been very aware that there is not enough detail to replicate exactly what was done. I often come away with the feeling of just being given the gist of what was done. I think this is a combination of papers that are too short, and possibly researchers wanting to maintain competitive advantage.

My impression on DL papers is that they are relatively clear to other papers in machine learning/math/stats. Especially for big conferences, many of the accepted papers have code released on github before the conference date either by the original researchers or by developers looking to get stars (usually googling paper and + github will help you here). A lot of these implementations end up turning into blog posts or being contributed to Theano/TensorFlow/Torch/Caffe/etc.

I don't see a huge competitive advantage in not releasing code. It's probably more beneficial to share ideas and collaborate. I imagine the reason it's not done more often than it is is because it takes time and effort to put this stuff out there.

Reply on news.ycombinator.com