This is a huge gamble IMHO. If you have access to LIDAR and RADAR, why not use them?
This means, LIDAR could just be additional sensor data augmenting the camera information. But it adds a huge cognitive load when combining the data. Besides the costs of the LIDAR system itself, it adds to the computational costs. Once the 3D-data from the cameras only is "good enough", you might get better results by spending the available compute resources rather on improving the analysis of the visual data alone. And the 3D data Tesla seems to be getting from the cameras seems to be pretty good already.
The big elephant in the room isn't to recognize a road sign correctly, it is to decide, whether it applies to the car or a different lane, road or whether it is actually a burger king logo :). There LIDAR might be just a distraction.
It will be interesting to see how the autonomic car race will pan out. Especially, when competitors are using different strategies. That als ensures, that not everyone is going into the wrong direction.
The gamble here is on the software, not the hardware. All the accidents that happen are not a failure of sensors not sensing but of software not detecting what the sensors captured. We're talking cars crashing into things on a clear day. Cars not recognizing situations that a human would recognize from the captured video. Etc. They need to solve that anyway. Regardless of whether they have lidar or not.
Other companies like Waymo pushing the Lidar agenda are also behind on their schedules to launch autonomous driving. But at least they have actual driver less cars on the road right now driving actual passengers around in very select markets.
Either way, this is more or less a greenfield market without a lot of contenders shipping actual products yet. So, Tesla can afford to spend more time developing this market. If they nail this somehow, eventually, it will be a big growth driver for them.
They probably started without lidar due to the cost (Tesla is one of few companies actually selling cars to customers, who are sensitive to price). The price of lidar has come down, but I would guess it requires a fair amount of expensive rearchitecture to add it in now, and they haven't bitten the bullet yet.
He was asked point blank if they would use Lidar as Tesla even if they were free... and he said no. In their opinion it's simply not the right approach to generate photons and read them as they come back. It's preferable to simply read the photons that are already coming in.
Then why have lidar? If it's only purpose is to agree with vision or you just discard it, then why even have it?
You might want to not have sensor B to lower the cost (as I suggested). Or you might want to have both sensors A and B because the provide some synergies (as gkop pointed out, better range of view, increased confidence, etc.). But it’s definitely not enough to just argue that the sensors may conflict.
I'm almost doubting my entire knowledge in the field now when all of a sudden everyone just accepts that "sensor fusion is hard" statement.
I just think there's more to the story than we're getting. Impossible to tell for an outsider though.
It seems to me that accurately determining signs, construction work, pedestrians, bikes, and even the intention/direction of these, is way harder than just reconstructing a 3D space from the cameras. I think the neural nets will have to do this internally anyway to be able to predict the direction a pedestrian is about to walk.
So I think LIDAR may be a benefit for limited self-driving in the short term. If it's a benefit in the long term remains to be seen, but there's good arguments that Teslas gamble will pay off, as they're allowed to use all their focus on the part of the AI that's most important for daily driving.
Just because something works for evolution/nature, it doesn’t mean that there isn’t a simpler/better engineering alternative.
We have spend hopelessly many years trying to build machines that walk — Now Boston Dynamics seem to be there. But it is really complicated compared to wheels, and their machines are not running at 2-300 km/h.
I think LIDAR is a very expensive part (~$10,000). Also, it is difficult to integrate with the car's design such that it does not make the car ugly. Tesla probably does not want to do a separate self-driving testing fleet, instead they want to use the existing Teslas on road to get training data. So they cannot use LIDAR in all cars sold to users.
I am sure if fully functional LIDAR units magically start selling for $100-$500 Tesla will start using them.
I’m not convinced. There’s good reasons they’re the lone player in the self driving space with this vision only play.
Does sensor B have enough other benefits to outweigh the costs of adding it? I don’t know, but that’s the actual crux of this problem.
Hence my question, "How much precision do you need to keep from running into a parked truck?"
But for radar, correct, those modules are available as a commodity for automotive lifescales.
The general problem is that simple, cost effective radar is low precision while expensive, complex lidar is high precision.