A self-driving toy car using end-to-end learning

In California, UC Davis holds a competition for track following robotic cars called NATCAR: https://www.ece.ucdavis.edu/natcar/

I helped assemble a Donkey car at the ARM event in Seattle last week, it’s really quite elegant in the efficiency of its approach. A couple hours from unboxing to self-driving if the top parts are already printed.

https://twitter.com/choongng/status/997339031586271233?s=21

I feel like a whole new world has just opened up before me. I have a rasp pi, what is the route with least technical barriers for me to get into this and start tinkering?

best github username. Mighty healthy!!!

In this case, the training is supervised. It’s a simple convolutional neural network that predicts the turning angle based on the image of the line on the “road”.

In the context of automated driving, 'end-to-end' tends to mean that the net learns a transformation function that maps input images to steering wheel and pedal positions. In that sense, there is 'no human input' at any stage.

In contrast to this, every significant AD effort uses hand crafted parts for most of their pipeline. Machine learning is mostly used in the initial object detection and tracking steps.

Wow, they seem to be using Hierarchical Temporal Memory rather than more traditional deep learning approaches. We've seen some rather grandiose claims about HTMs' capabilities from Numenta (who I think invented the approach?) but no really convincing applications yet, so it's really interesting to see a concrete demonstration like this.

From that article:

> Compared to explicit decomposition of the problem, such as lane marking detection, path planning, and control, our end-to-end system optimizes all processing steps simultaneously. […] Better performance will result because the internal components self-optimize to maximize overall system performance, instead of optimizing human-selected intermediate criteria, e. g., lane detection.

One can imagine that it might be more difficult to get a network to solve this large problem all at once, and that there might be easier to decompose the problem and solve each part. Would it be a good idea to guide the end-to-end system by first decomposing the problem and solving each part, then using that solution as a starting guess for the whole problem? I mean, the decomposition might perhaps be a reasonable approximation of how the whole problem should be solved. (Then again, it might not.)

You need to be more specific about what you want to build.

A true LIDAR system infers the round trip time of photons, either directly (time-of-flight) or based on some proxy like phase or frequency shift (AMCW/FMCW) LIDAR.

Lots of people include triangulation systems, which I disagree with. For example the RPLidar A2 is just a nicely packaged version of Kurt Konolige (et al)'s RevoLDS [1]. It projects a laser spot and takes a picture of it. Add in some known geometry, and you can use triangulation to measure distances. It's easy to build, you can do it with any old camera and a laser pointer. It's incredible how overpriced the A2 is, given that the Revo was published as $30 system.

A true LIDAR is much more difficult to build. If you go time-of-flight then you need a good laser and excellent timing electronics. If you go phase-shift then you need beam mixing/separation optics, beam modulation and a phase detector. Neither of these are easy for the average hobbyist. If you're interested, buy a laser tape measure from Leica and have a look inside to see how it works.

I belive the LIDAR-Lite is a proper LIDAR system - it ranges up to 40m which is a good hint that it's probably a true time of flight system. The Scanse Sweep is a LIDAR-Lite turned into a 2D scanner, like the sibling post. A good indicator is price. Real LIDAR systems are expensive - starting at £1k typically for something like a Hokuyo. They also tend to be much better built than the hobbyist stuff.

In principle all you need is a pulsed laser, a big collecting lense, a fast rise-time photodiode and a good (picosecond accurate) timing circuit.

[1] https://www.robotshop.com/media/files/PDF/revolds-whitepaper...

Presumably the file is a recording of a human operating the car?

The steering angles are probably automatically obtained from the script you run when manually training the toy vehicle. I presume at every frame an image is captured along with the steering angle for this image.

You then create a neural net architecture that is being fed images and steering angle positions and outputs steering angle predictions. This is a regression problem that can be expressed in plain English as:

"First train the neural network with a collection of images and associated steering angles. After training, if I were to give the neural network a new image it has never seen before, what steering angle would the network predict?"

NVIDIA has a paper on it[1], and I blog about a similar coursework I complete as part of Udacity's self-driving car nanodegree[2]

[1] https://arxiv.org/abs/1604.07316

[2] https://towardsdatascience.com/teaching-cars-to-drive-using-...

Self-driving cars are already reasonably good most of the time. They need to get better at specific cases, which would be a lot harder to model in a toy. Stopping distance is going to be a lot different, just for one example.