How are Waymo managing this? None of their competitors appear to be even close.
But I suspect this means Waymo's software is ultimately more risk adverse. If a Tesla stops in the middle of the road then the customer has to take action, which is frustrating and makes the technology look bad, so there is strong incentive to remove that frustration even at the cost of safety. If a Waymo stops, the remote operator has to take action, and the customer can keep staring at their phone without being particularly affected - it just seems like the Waymo is "thinking."
1. They've been doing it for ages. They had cars on the street fifteen years ago.
2. They bet on hardware that's not just cameras. Cameras—in practice—are still not the best tool for the job. Cameras see in 2D, they get dirty, they are easily blinded and obscured by dirt, etc.
3. They have data from every Google Street view and mapping car ever deployed. They have the most data and the most current data. Every Tesla on the road would need to be maxing out its LTE connection all the time and they still wouldn't have the breath and quality of data that Google has.
4. Google is throwing money at Waymo. They can see the potential profit if they win. They're not going to get dumped like Cruise.
Tesla went from very expensive cars down to cheaper ones. It would make so much more sense to do the same for perception. First go over board and go for high bandwidth input and lots of processing power and optimize later.
I used to regularly have to make a left turn onto a rural highway on foggy mornings. Sometimes people drive faster than they should in fog. Sometimes fast enough that by the time they could see I'm in the intersection turning they would be too close to stop.
Cars going fast enough to have that problem made enough sound that they could be heard quite a bit farther away than they could be seen. I'd open my windows at the intersection and listen until I couldn't hear any highway traffic. Then I'd know that any approaching cars are far enough away that I should have time to turn onto the highway and get up to speed before they arrive.
IIRC I think it’s the section (1:23:25) – Camera vision
The TL;DR is that sensor fusion is really hard, and their bet was that keeping the training pipelines simpler would let them scale faster/easier, and human vision is the existence proof that it can be done without lidar.
Human vision is an existence proof for human-level performance without lidar, but Waymo is an existence proof for 10x human performance WITH lidar. Right now the latter is where the bar is, and it'll keep being raised. I don't think at this point one could get away with deploying AVs at scale that are significantly less safe than Waymo.
Also: if sensor fusion is so hard, why is Waymo able to solve it but not Tesla?
I think Karpathy's point is that Tesla wants to try to avoid the "entropy" that comes from adding a sensor (senior software engineers and higher understand this concept). Every sensor (and every version of it -- sensor hardware does get updated) you add requires recalibrating the software stack, the hardware design, which introduces points of failure every time you roll it out.
According to Karpathy, Tesla does use Lidar -- but only at training time, as a source of truth. Once the weights are learned, they operate without the Lidar.
Have a full sensor suite may work for Waymo at the current scale (limited cities), but scaling beyond that poses problems.
Whereas Tesla has to work with a different set of scaling economics -- that of a mass market vehicle already deployed globally.
They were doing it first by years and have been spending the most on it throughout so not surprising they'd be ahead. Even now they've probably got several times more employees working on autonomy than Tesla
Compared to people who drive pretty poorly most of these will probably do better
"Lidar is lame, stupid"