How to Learn Machine Learning and Deep Learning: A Guide for Software Engineers | Hacker News Reader

How to Learn Machine Learning and Deep Learning: A Guide for Software Engineers

renanmf.com

59 pointsrenanmoura14 comments

While interviewing candidates I have observed an influx of candidates having a strong bias towards neural networks in the recent years - i.e. not understanding concepts such as overfitting in general and telling me how dropout would fix that.

We are a mid-size AI/ML consultancy company. We have not yet have a project where we'd actually been given enough data and resources from a customer to be even able to learn a NN from scratch. Vast majority of the problems we tackle are easily solvable with traditional ML methods which these fresh candidates consider obsolete and boring.

This tutorial pretends to be mildly aware of that, yet the first course it offers is Andrew Ng's ML, which is structured in a way that it culminates in building a NN, a brief mention of RF and then it jumps straight into DNN.

A disclaimer that this is hardly a thing a person would actually do for living should be included at the beginning of every tutorial such as this one.

Hey, author here.

I wrote this back in January and I am aware of what you are saying about people rushing too fast into DNN.

The article is an answer to some people asking me for general references.

But I agree with the disclaimer in general and will incorporate it into the article.

BTW, I started with Ng's course simply because of the way he teaches in a very clear way, and not scaring people out, it is a very good intro and most people would feel confident in moving forward to other resources after his course.

I started working through the ISLR book then found someone else to do it with.

http://faculty.marshall.usc.edu/gareth-james/ISL/

You can download the PDF and there are videos too:

https://www.r-bloggers.com/2014/09/in-depth-introduction-to-...

Doing the the questions and labs is helpful with a partner.

Machine Language Tokyo has several good sessions. Not many times that work for me, unfortunately.

https://www.meetup.com/Machine-Learning-Tokyo

A bit OT: I've always wondered - you see a lot of relatively young engineers and devs doing work in what seems to be cutting edge DL work, and some of these are even self-thought, with sometimes only a few years of experience (at least if their open github accounts and blog resumes are any good gauge of experience).

On the other hand - the only (really) talented people I personally know, that are working on Deep Learning problems, all have heavy and very solid academic backgrounds from Math / Physics / CS / etc., and are either finishing up their Ph.D's, or were done the past few years. Most started studying (or working on) ML before the Deep Learning craze took off. All in all, these guys have at least a decade of solid experience - but in return, know a metric ton of classic ML / stats / grad. level math / etc. by heart.

Obviously, in probably most cases, one can't compress 10 years of hard work experience into 1-2 years, even though that's what a lot of these "roadmaps" seem to promise.

Makes me wonder if the internet-world of deep learning is filled with savants that are able to catch up with decades of research and best-practices in only couple of years, or if these are people that are really, really specialized in some niche application of DL.

Seems to me humans are really good at 'catching up' based on what we perceive as possible.

For example when a new world record is set in a sport often the idea of such a performance is possible raises the bar for the entire field rapidly. Within months multiple competitors are nipping at the heels of the record.

I also see this in video game speed running. When a new breakthrough record is set suddenly the competitors begin to catch up to the new record now that they see it is indeed possible. Even new people playing the game don't take 10 years to catch up skill wise when they can learn from previous attempts from the 'old masters.'

It seems like even in the arena of AI breakthroughs as soon as some paper gets published others quickly catch up. I do agree there is no replacement for 10 years of research and work, but future people studying in the field can and do catch up.

It seems cutting-edge how? Because for example you can literally go to GitHub and get a project with a complex deep learning architecture close to the state of the art (or at least last year's state of the art) for some things.

Then pull that up on your screen and start messing with the structure of the graph, adding layers etc. with impressive sounding names like "relu" etc. Lol. It could look cutting edge and it actually could kind of be cutting edge as far as the fact that regardless of your background the next step to making an incremental improvement in an area may be taking a known model and tweaking the hyperparameters and assembling a good dataset.

But obviously there is a difference between that and creating a new architecture or approach from scratch. In other words, you are not going to invent GANs or capsules by tweaking hyperparameters. Or come up with other more basic innovations necessarily without a deep understanding of the underlying concepts.

However I will say that the repeated practical application of deep learning seems like it is going to be a critical part of a having a strong grasp. And someone with a Ph.D in ML mifht actually be less useful for a particular application than a person who has actually used Keras for ten different real projects.

At the core, many people are involved in applying existing approaches by assembling datasets and tweaking hyperparameters and architectures. Which maybe amounts to modifying a few hundred or maybe dozen lines of Python code that describe the different layers and learning rates etc. and load the data.

People can leverage programming libraries and be more effective than another guy who has less experience with that library but is a true expert in statistics and LaTeX.

I feel like it's more complementary then that. You need the PhDs to develop core theories and technology, and you need the young engineers and devs to understand it enough to apply it in other domains.

Both can be described as doing "cutting-edge DL work" but just in different domains. And, the young devs don't need to try and catch up, they need to be generalists.

There's a huge difference between being able to apply a set of known techniques and being able to create novel techniques. I suspect most of the autodidacts you are referring to are able to read papers and implement them, and are well-versed in the practical aspects of how to tune models, but would likely struggle with doing something novel enough that is actually worth publishing.

There are exceptions, of course, but for certain types of work that extensive background in math/stats/cs is absolutely necessary.

Trailblazers will always make slower progress than those following in their footsteps. It took researchers how long to develop calculus and we become effective at wielding it in 3 classes

Ng's Deep Learning Specialization is awesome but I feel after this second course I need to stop and create my own exercises to build and apply NNs from scratch because I am learning how to convert math notation to vectorized Python code as much as anything else. I mean I am learning some core concepts along the way which I definitely appreciate and the reason I am continuing, but to become proficient I need to apply them without the hand-holding. Because those Coursera assignments have evolved to be computer-graded and easily approachable and so they actually do most of the thinking for you.

Also I feel the modern approach is not vectorized Python but something like Keras or Tensorflow 2 or anything that is maybe higher level or will allow me to keep the computation on the GPU/TPU and not going back and forth to the CPU like I assume numpy must be doing.

Do NOT trust any advice that claims you can learn math by watching a YouTube video. Math is not a spectator sport!

what do you mean

you watch the video and then you do your homework

you just learned math via youtube video (and your homework)

Bit of a nitpick, but it should be "Artificial Intelligence piques my interest" as opposed to "picks".

thank's for the nitpicking, fixed to "piques". :)

Reply on news.ycombinator.com