MIT 6.S094: Deep Learning for Self-Driving Cars | Hacker News Reader

MIT 6.S094: Deep Learning for Self-Driving Cars

selfdrivingcars.mit.edu

158 pointskennethko24 comments

There seems to be this assumption in the educational community that deep learning is the answer to automatic driving. Yet that's not how Waymo does it. They're geometry-first. Most of the control is based on "flat road here, big obstacle there", plus map data. Machine learning is only for "what's that". See the TED talk by Chris Urmson.[1]

Driving on deep learning alone will work great, most of the time. Some of the time, it will do something completely bogus. Like Tesla's self-crashing cars, which use Mobileye's system to not-recognize obstacles which block part of a lane.

Sometimes, the system will sense an obstacle and won't be able to identify it. That has to be handled safely.

[1] https://www.ted.com/talks/chris_urmson_how_a_driverless_car_...

AFAIK most SDC companies are using all kinds of control inputs, whether they are based on classical Computer Vision (RANSAC, HOG, SVM), on Deep Learning with 2D convolutions, on Deep Learning with 3D convolutions and RNN (2D + time on a series of images), Deep Reinforcement Learning (lane keeping, realtime path planning/obstacle avoidance), LiDAR+Radar+Ultrasonic combo via Kalman or particle filters, GPS, IMU etc. and then mix all those signals together to figure out which ones are the most probable. And this all has to happen with super-low latency. With the latest NVidia 500W GPUs for cars we might get far better precision than what was possible with Jetson TX2. You are right Deep Learning alone won't make it reliably, as sometimes output goes completely wrong and simple filtering/averaging past few frames might not be sufficient to prevent your self-driving car becoming a self-flying car. But combining all these inputs together, maybe via another DNN, seems like the best way forward.

Yes, we are using a fusion of different types of sensor input. I have to be careful to not divulge trade secrets, but I would also add that sensory input (deep learning based or otherwise) is only the very first step in a long and complex chain of tools, algorithms etc. The media and academia are focusing almost exclusively on that step, but it's really only a small piece of the puzzle we are working on.

You are also correct on the performance issues. The DNNs you would require for the very complex decisions are currently too large to be used on automotive ECUs. But we also still have big open questions regarding provability of safety when using neural networks that need to be answered before we can use them for core decision making algorithms.

TBH it seems more like academia is jumping onto the huge wave of hype that has been rolling these last few years. Lately we (a company working on automated driving amongst other things) have seen a lot of interest in internships, thesis topics etc. from students, but only as long as it includes deep learning. It's no surprise to see academia cater to that demand.

University of Montreal also has an autonomous robotics class called Duckietown (formerly at MIT). Having taken it last Fall, I can say it's pretty comprehensive. We covered a bunch of related topics from computer vision, deep learning, filtering, SLAM, with guest lectures from industry researchers and roboticists: http://duckietown.org

If you're in Montreal, we're having a public demo day next week. http://diro.umontreal.ca/departement/evenements/une-nouvelle...

This is the course home page for the YouTube video discussed briefly here 2 days ago: https://news.ycombinator.com/item?id=16157843

Thread winner goes to bitL summarizing the differences between several similar courses as he also has in this discussion:

>bitL: TL;DR: Once you finished Udacity, MIT gives you more [...] you'd understand instantly (except for Deep Reinforcement Learning

Apparently the chance to order this year's class t-shirt has been extended to Monday!

So nowadays, even MIT is a vocational school? This just seems too damn specific to not be a direct industrial request.

edit: corrected per bitL

Expect more of this as society continues to devalue degrees that don't tie directly into a lucrative professional career. College is expensive, with tuitions not anchored by a limited credit market. People will rationally turn away from fields of study that won't let them escape from their unforgivable debt.

The joke will be on society after half of this stuff gets refined down to ultimate simplicity (like bicycles), and the other half is refined to a point of "good enough" (like terminals) where no one cares anymore.

MIT is not Ivy...

This isn't a full course, it's an IAP course. IAP courses at MIT are special short courses, often student-run, between the fall and spring semesters that are supposed to cover specific, often practical topics in order to spice things up between the main classes.

It seems to me that this is a survey course of a recent cutting edge research area with hands on projects. And MIT is an engineering school, which so an applied course like this isn't out of place.

Also, it's IAP (Independent Activity Period), which is a one month break between semesters. That's just enough time for a very specific course. There's another very specific IAP course in special relativity, for example.

Besides, the topics they're covering here, like deep learning, SLAM, image segmentation, etc are useful in many other areas.

...It is open to beginners and is designed for those who are new to machine learning, but it can also benefit advanced researchers in the field looking for a practical overview of deep learning methods and their application....

Someone who took this course before, is it really good for beginners? Also, the course title suggest it would deal with models for self driving cars. Description says "practical overview of DL methods and their application". How specific is the course ?

Why not? It's an introductory course...everyone has to start from somewhere and the most experienced experts in this field only have a few years experience given the newness of the field in general. Don't underestimate yourself!

I'd say it's for intermediate level. Let's say you have already finished 1-year long Udacity self-driving car nanodegree and reached intermediate level, then these lectures will be understandable to you. If you are total beginner you are going to be lost, but if you are an MIT newbie, you might be fine with some work in your spare time. To understand some subjects like Deep Reinforcement Learning you might need some related graduate-level coursework under your belt. It's not bleeding edge though, so researchers might think the lectures are trivial.

Dang looks like the class already started.

Would it be worth it to formally enroll in the class if you're trying to get into the industry?

why would someone pick a guy who did this over an MS grad from Stanford

lol i'm not sure if you are being sarcastic

You've posted a ton of unsubstantive comments to HN. We eventually ban accounts that do this, so would you please stop?

It's an important question, though. I see a lot of interest in MOOCs, the Udacity nanodegree etc. on the internet, but I can't see my employer/me hiring someone whose only qualifications are online courses. I know absolutely no one in my entire department who does not at least have an MS. And we are doing exactly the jobs that people who take these courses want to get hired for.

I like to do MOOC courses to get an overview of / intro to a certain topic I am interested. But once people charge a substantial amount of money for a degree, it is absolutely pertinent to question whether the degree is worth it. E.g. IMO udacity is very close to being fraudulent with the promisses they make in their advertizements of their nanodegree.

Ok, but an important question deserves a substantive comment.

Shameless plugin: Here is a recent presentation I did while finishing Udacity's Self-driving Car Nanodegree. It might help with some of the concepts, show you what was Udacity teaching and complement the material from MIT:

http://bit.ly/2EOKIXy

Have fun!

Kind of disappointing to see it is about cars not for cars. Kitt and Lighting Mcqueen must feel kind of left out.

What about Herbie? Just because the Love Bug doesn't have blinky lights or smooth Pixar animation doesn't mean he's obsolete!

...We might want to think twice about admitting Christine though.

Reply on news.ycombinator.com