6.S094: Deep Learning for Self-Driving Cars | Hacker News Reader

6.S094: Deep Learning for Self-Driving Cars

cars.mit.edu

327 pointstancik51 comments

Trying to do self-driving entirely by deep learning in a reactive system seems like a terrible idea. Deep learning to classify objects (bicycles, pedestrians, cars, traffic lights, telephone poles, cops) is fine. But map building, planning, and obstacle avoidance needs to be more reliable than a purely reactive system can do.

Look at the videos from Urmson's talk at SXSW. That shows the worldview of a Google self-driving car. It's about 80% geometry and 20% classification.

Yes, you can get a pure deep learning system to drive on a freeway. But how does it do in a more cluttered environment? A system that builds local maps and profiles terrain with LIDAR can deal with clutter.

Totally agree on this. Making neural networks build a debuggable scene graph is a pretty hard challenge. With the scene graph things become quite easy.

Why exactly can't you use deep learning to drive in a "cluttered environment?" You didn't mention why it's a terrible idea (I have no background in deep learning, just curious)

Both model-based and model-free reinforcement learning methods are useful and sometimes are combined in an agent.

What are good online sources for a beginner to learn calculus, linear algebra and stats/probability needed for machine learning?

I don't want courses that teach math by coding but actual math courses. A lot of these online ML courses treat math as a black box so you never fully understand what's going on & a lot of math based ML courses are too advanced.

So I'd rather focus on understanding the math basics before diving into ML. So what do you guys recommend?

Why not a book instead like the elements of statistical learning?

http://statweb.stanford.edu/~tibs/ElemStatLearn/download.htm...

Based on Amazon reviews recommending this book to a beginner is like recommending rudin to a high school junior that wants to learn calculus for the first time. Is there a better book?

Introduction to Statistical Learning? It's free and has an available MOOC http://www-bcf.usc.edu/~gareth/ISL/

Thanks for the link.

https://www.amazon.com/Elements-Statistical-Learning-Predict...

https://www.amazon.com/Introduction-Statistical-Learning-App...

The former seems to assume prior knowledge and the latter seems more beginner friendly, except it uses R (in case anyone wants the distinction).

These books assume some degree of Linear Algebra, ESL is definitely not for a entry level.

I would like to add www.codingthe matrix.com for any developers, which has a MOOC as well (Coursera), as a complement to Strang.

R is well worth learning if you're going to be doing any serious work with data. It's a weird-ass language, but it does data analysis/graphing/modelling really, really well.

I support this statement. Not a book for beginners, IMO

This is in no way appropriate for someone who is asking for math basics.

A lot of the material in this book makes much more sense having had experience coding machine learning algorithms.

Linear Algebra by Gilbert Strang, MIT https://www.youtube.com/watch?v=ZK3O402wf1c

Probability by Joe Blitzstein, Harvard http://projects.iq.harvard.edu/stat110/youtube

Excellent, thanks for you recommendation!

Gilbert Strang is an excellent lecturer and teacher. He, along with Andrew Ng, are amongst the best.

Calculus, by James Stewart. I recommend you buy a dead tree copy of an old edition (I paid $5 for mine, shipped. I don't know how they made money off that sale.) Might not be the best book, but it covers everything you need to know, has good exercises, everyone uses it, you can find the solutions manual online, and the harder problems have an answer on stack overflow.

And some linear algebra book by Strang (as others have said.) I don't remember which one I learned from, but I do remember it was excellent.

I can't recommend a good stats/probability book. People recommend Statistical Inference by Casella and Berger, although I wasn't super impressed with it.

You'll also want a good understanding of set theory and proof writing. I don't remember what book I learned from, but try to find some introduction to math thought.

This is all assuming you have a good understanding of algebra. The above mentioned books require geometry/trig, but you don't actually need that for machine learning.

> Calculus, by James Stewart

I learned from the Stewart book and it's not terrible, but I don't think that is a very good book for self learning. How to ace calculus or the calculus lifesaver, plus a book with solved examples is probably better for self study.

This was my university calculus textbook and I liked it well enough. Granted, I was learning with the assistance of lectures to teach me the concepts too so I don't know how you'd fare with only this textbook, but I liked it well enough to drop coin on a physical copy.

My philosophy with textbooks was to buy only ones I felt I'd go back to after uni - basic references in each major field I studied - and this was reasonable for me for calculus. My only regret was shelling out for new copies because I was young and stupid and thought I needed them right now for the class, before I knew better. You can certainly pick up cheap second-hand copies per the parent post.

I also used Kreyzig too but found that to be a bit harder to learn from, albeit more in depth.

I'd also recommend Calculus for the Practical Man. It's what Feynman used and is quite good for learning how to calculate various things with calculus.

Incidentally, my college calculus class also used Stewart.

Google "MIT OCW Scholar".

Sites like Khan Academy, PatrickJMT, Paul's Online Math Notes, and BetterExplained are useful, too.

Coursera and edx also have offerings.

Khan Academy is an amazing true MOOC resource. The calculus goes up to early undergrad. The differential calculus is the most important for ML, in my oppinion.

Probability and statistics will help ML as well.

I would recommend Khan Academy before any other resources, whatever your level. It has online exercises with auto-correction, and a mastering system that uses spaced repetition. This will ensure you'll actually assimilate the material. After you can use another resource as a reference.

Ocw and Gilbert strangs material is a must for linear algebra. For beginners, I enjoyed Stewart calculus.

For linear algebra this youtube playlist called essence of linear algebra is probably the best intro I've ever seen. it's relatively short (like 2 hours) but gives you all the right intuitions

https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...

Step 1: buy a Car

Step 2: duct tape a lidar to the top of the car

Step 3: ????

Step 4: enjoy your self driving car

BTW, there's a well regarded new book on autonomous cars that looks like fun: 'Driverless: Intelligent Cars and the Road Ahead' by Lipson and Kurman. Lipson is a professor of robotics at Columbia.

https://www.amazon.com/gp/product/0262035227/ref=oh_aui_deta...

Which specific Coursera and/or Udacity courses are good for coming up to speed on the math/stats&probability knowledge needed for their SDC/AI/ML courses?

I'm curious too, I'd like to know what prerequisites are needed for these lectures to make some sense and be beneficial.

A linear algebra course is the bare minimum (and frequently it can be enough). You can pick up any textbook and that would give you good enough preparation.

After that you probably will have enough domain knowledge to choose what course to take next yourself.

If I don't really remember what all was in my Linear Algebra class from 15 years ago, but have a working knowledge from graphics would that work, or would I need more of a theoretical background in linear algebra do you think?

I recommend Coursera's Coding the Matrix (linear algebra and a bit more) http://codingthematrix.com/ and https://cs.brown.edu/video/channels/coding-matrix-fall-2014/

And edX's Intro to Probability (MIT 6.041x) https://www.edx.org/course/introduction-probability-science-... BTW, this course starts again on Jan 17.

I don't know any good MOOC statistics courses. Most that I've seen have oversimplified the concepts into recipes.

For math/stats&probability in the specific problem domain of autonomous vehicles (of which self-driving cars are a subset) I'd recommend no other than Sebastian Thrun's Udacity course "Artificial Intelligence for Robotics" [https://www.udacity.com/course/artificial-intelligence-for-r...]. The content is practical, and Thrun, in my opinion, is great at breaking down complicated topics (e.g. Kalman filters) into simpler sub-components and explaining the intuition behind them. You'll get a heavy dose of math/stats&probability in the context of autonomous robots (and in certain cases, Thrun contextualizes to his work in the DARPA grand challenges while at Stanford, which laid the groundwork for his work at Google on the self driving car project [now Waymo]). I can't speak to how much the "traditional" localization, mapping, and planning techniques taught in the course crossover to deep learning approaches, but you'll no doubt get the math/stats&probability knowledge you're looking for, in the context of self-driving cars.

It looks like they'll be posting the lectures online which is pretty rad. I'll be sure to make some time to watch these.

I was reading through the page on the Docker install [0] and after giving it a look I'm really interested in a docker GUI. I'd hate to have to type or memorize that for a class or for doing something useful and I'd also dislike needing to maintain a file with 50 different scripts to stop & start docker and what I want to run at any given time.

Anyone know of something like that?

I wonder how it compares to Udacity's similar course.

IMO, the udacity SDC course is pretty bad.

I just enrolled in the Udacity SDC nanodegree. Can you elaborate?

It's pretty cool, you'll have fun! It's "practical" not "theoretical", so the focus is on getting code working and understanding bare minimum of math you need.

Sounds pretty similar to Udacity's AI for Robots course! Good to know that they focus on intuition rather than rigor. Many professor enjoy stroking their ego with rigorous math, rather than communicating higher-level ideas and making their subject seem "easy".

AI for Robotics is a way way simpler course than SDC ;-) In SDC you actually get to train CNN to classify traffic signs from the scratch, to make a car in a racing game learn to drive similarly to your style and to detect real-world road features properly as you are driving. It made me smile very often when I saw how simple they made it appear :)

Of course haha! I was remarking on the breadth by which they explore and prove the underlying math.

That perspective seems a little naive.

They are professionals, so they know "what it takes" to be good at the subject.

Machine learning math is not rigorous in comparison to the type of math you'd study as a math or physics major.

Machine learning math is useful precisely because of the fact that it shows you intuitively how the concepts work.

Knowing the "high level" ideas is near-useless. I can get the "high level" ideas of quantum mechanics from any high school or pop science physics book. Does that mean I know quantum mechanics? Only at an extremely superficial level.

And I suspect anyone taking a MOOC on AI or machine learning wants to end up with more than superficial knowledge.

Sorry I wasn't clear about "high level".

I was mostly speaking about math professors in college. I have degrees in Math and EE...advance math isn't an issue.

Having abstract intuition (aka "high level ideas") in math is incredibly value.

Sometimes things get hairy in the computation and you need to step back and consider the bigger picture again.

If you don't understand how the math works then you don't have the intuition and vice-vera.

@bitL's comment of it being "practical, not theoretical" is pretty spot-on. I'm currently enrolled in and taking the course; I've also taken Udacity's CS373 course (when it was first offered in 2012) - and early courses that formed the foundation of Udacity and Coursera (ie - the Stanford-sponsored AI Class and ML Class, respectively).

Again - what I noted in another comment - if you aren't comfortable with coding (Python and C/C++ in the nanodegree course), or linear algebra - you might have a tough time. You'll also likely want to brush-up on your stats/probability knowledge (though this hasn't come up yet - not like the CS373 course, or the AI and ML Classes). Finally, while you won't absolutely (well - not that I've seen yet) need super-detailed knowledge of what a derivative is or how it is derived (?) - that bit of Calculus seems to be important in machine learning (it is something I struggle with).

Ultimately, give it a shot and have some fun; so far, I have enjoyed everything (just finished the lesson and lab for the Keras framework last night). Don't get too hung-up on the "due dates" for projects; ultimately the main due-date is the end of the term (you have to have everything completed by then to not have to retake the term). Be sure to use the forums, your mentor, the Slack channels, and the facebook page to get help and inspiration. Oh - and you are able to publish your solutions and code around, but if you decide or need to use code provided by someone else (or you use someone else's solution, or ideas from a paper, etc) - YOU NEED TO DOCUMENT THIS in the project rubric (otherwise it will be considered plagiarism which, if found out, may get you kicked from the course) - so be sure to credit your sources!

Finally - if you enjoy the course - be sure to check out Udacity's CS373 course, their "Intro to Artificial Intelligence" course (both of those I believe are free?), and Coursera's Machine Learning course (also free). There are also a ton of other great online resources and books out there; let me know if you need or want suggestions!

On mobile so this won't be thorough, but compare the udacity course to the free (!) Coursera course from Andrew ng and you'll see the difference.

I'm not even talking about mathematical rigor, I myself prioritize intuition over symbolic rigor, but you need to have the intuition of what's going on, not treat keras like a black box that magically spits out a ConvNet.

Another thing I dislike is the lack of real world production deployment work. In an engineering oriented course, this should be the top focus. Unfortunately, this isn't really the case.

That being said, I would say it's better than many universities.

How so? And are you talking about CS373 or the Nanodegree course?

Mind you, I'm kinda biased as I have taken the CS373 course, and I am currently enrolled in and taking the Nanodegree course.

But I am willing to be objective about it. I can honestly say that in both, I have learned some useful things. What I have learned I believe I can build upon myself to explore new areas (for myself - maybe not in the general sense). Are the courses perfect? No.

Both skim rapidly over the material - which can leave you wondering and questioning things. In many cases (labs and projects especially) you need to rely upon asking questions of others - your peers or mentors - in the forums and other outlets (I particularly like the Slack channels in the Nanodegree for this). There's more than a bit of "reading in between the lines" and needing to reference notes and prior code implemented. But that should be expected.

I think both rather expect you to already have a good handle on coding, as well as experience or exposure to linear algebra, stats/probabilities, and some calculus (primarily what a derivative is and how to get one); those last two are sticking points for me (especially the latter - prior courses have at least given me a better grasp and intuition about probability - but I am not an expert at it).

If you don't have those pre-reqs, you're going to have a tough time at progressing meaningfully thru the courses.

I also know that neither course is a substitute for anything like a uni-level CompSci degree, or courses for that, or electrical engineering, mechatronics, machine learning, etc. If you want that kind of rigorous treatment, you're going to need to buckle down and do it.

But you know what? These courses (and a few other things I've read) are inspiring me to pursue a BS, then an MS in CompSci! Which I think is great, even if it's a "late-stage" thing for me to do (I'm 43 years old, and I never went to college - my higher-learning after high school was an "associates degree" from a private tech-oriented vocational school which I am pretty sure is worthless today; I've taken a couple community-college classes, too). I plan to do this all online (I don't have the time or money to do it any other way, unfortunately).

Right now, I'm not sure how these Udacity courses will fit into things, but I'm using them as stepping stones to greater possible efforts. The worst case scenario is that I use what I have learned in my hobby of DIY robotics; I've spent more for less (for instance, a year-long membership to TechShop that I only visited about 3-4 times, because it was located so far away from my house).

So - I guess I am curious about your views and why you think it is "pretty bad"?

I'm considering taking it as well. Any more feedback would be greatly appreciated?

Really? I've heard only good things. Why is it so bad?

super cool to see this here. do the lecture videos go up the same day as the lectures?

They should be posted within a few days of the lecture.

It irks me, that this site is not using HTTPS. And wants me to register using passwords

Reply on news.ycombinator.com