If the goal is to be safer than a human driver then it will require better than human sensors, such as lidar. Camera only approaches will not stand the test of time.
The concept that biological systems have made 3D vision, navigation, and object avoidance work without LIDAR is certainly attractive.
But there is a LOT more to it than just a photosensor and a bunch of calculations. The sensors themselves have many properties unmatched by cameras, including wider dynamic range, processing in the retina and optic nerve itself, and more, and the intelligence attached to every biological eye also is built upon a body that moves in 3D space, so has a LOT of alternate sensory input to fuse into an internal 3D model and processing space. We are nowhere near being able to replicate that.
The more appropriate analogy would be the wheel or powered fixed wing aircraft. Yes, we're finally starting to be able to build walking robots and wing-flapping aircraft, and those may ultimately be the best solution for many things. But, in the meantime, the 'artificial' solution of wheels and fixed airfoils gets us much further.
Ultimately, camera-only vision systems will likely be the best solution, but until then, integrating LIDAR will get us much further.
Why though? How could it possibly be better than camera plus other sensors?
For example, several drivers of Tesla vehicles have been beheaded when a semi-truck turned/crossed in front of them and the car on autopilot evidently identified the white side of the trailer as sky and drove right under it, removing the roof and the occupant's heads. LIDAR would have identified a large flat object at range decreasing at the approximate speed of the vehicle, and presumably the self-driving system would have taken different action.
edit: I think perhaps you didn't quite mean it that way, but it sounded like you were saying "eventually, camera-only will be superior to any other possible system, including camera + other sensors", which just seems nonsensical.
To your point, there are MANY situations where that will never be true, where the extra 3D info will absolutely make a difference.
Going back to the wheel/leg and fixed/flapping wing analogues; legs and flapping wings will likely always be superior for rough terrain and tight spaces. However, legged vehicles will never be as fast as wheels can go on smoother roads, and similarly, flapping wings will never be superior to fixed-wings+power for long haul or heavy air transport.
So, you're right — it's Horses For Courses — different solutions will work best in different situations.
Like echolocation in bats and dolphins... Excellent point!
In fact, humans do have some echolocation capability [0,1]. That should tell us that LIDAR (or emitter-receiver-range-finder capability) may ultimately always be a core piece of the solution.
[0] https://en.wikipedia.org/wiki/Human_echolocation
[1] https://www.smithsonianmag.com/innovation/how-does-human-ech...
Having the same sensors as a human but being more attentive would be a step up. That said, I think camera-only is not good enough for now.
Peppering a few webcam-quality cameras around a car and plugging it into an Intel Atom processor probably won't be better than our eyes and brain, even if the cameras don't blink or get tired. It's only going to get better though.
...only if you count the field of view you get from moving your eyeballs. You wouldn't say a PTZ camera has "360 FOV" just because it can rotate around. The "576 megapixel" figure is also questionable. Peak resolution only exists in the fovea. Everywhere else is blurry and much lower resolution. You don't notice this because your eyes does it automatically, but the actual information you can receive at any given time is far less than "576 megapixel".
And, vision can't work in/penetrate heavy snow or fog, which is transparent to radar.
Vision is an indirect measurement. Lidar/radar is a direct measurement. I'm curious if there are any other safety critical systems that uses such massively indirect measurements?
Does a truly featureless wall/road with no visible edges actually exist in the wild? I’d expect cameras with high enough resolution, spacing, and FOV would handle any real world examples but maybe I’m wrong.
At this point it does not feel safe to trust self driving cars or assistive systems designed, built and tested primarily in California or the southern US for one simple reason: they do not get the range of adverse weather conditions that drivers in the rest of the world have to deal with and adapt safely to on a regular basis. It's easy to make a self driving car that "works" on California style freeways which are almost never under construction because they don't wear out as fast. In other places like eastern Ontario we sometimes have to deal with temperature shifts from -30C to +10C in 24 hours, salt our roads like crazy in the winter, and have a much wider range of typical weather conditions. These all take a significant toll on road infrastructure, and mean that what are rare corner cases in California become regular events elsewhere. We have 2 seasons where I live: construction season and winter. Based on several published reports of self driving cars hitting parked emergency vehicles or lane confusion in construction zones, I simply do not trust that the current widely available "self driving" vehicles are provably safe outside of the near ideal conditions present in California.
At least Waymo seems to be quite hesitant about rolling out to cities that have less favourable weather.
What I would like to see is for regulatory bodies with a safety first approach to accidents (similar to how the FAA investigates and regulates commercial aircraft) be involved in setting the criteria for the design, testing and regulation of self driving cars and driver assistance systems. Reading reports and watching shows about the root cause analysis of airplane crashes is fascinating, and it shows just how hard it is to learn how to make large and complex real world systems safe. It has taken plenty of deaths to get us to the point where commercial flights are safer than the trip to the airport, and it will take many more deaths before self driving cars are appreciably better than humans. Some of the most important lessons from aviation are about the interaction between pilot(s), crew and automation, and how those systems fail.
Test cases / data for self driving cars should be shared and made public. If we're trusting our lives to a piece of software, we should be able to see how well it does across standard test cases that the industry has encountered and developed, and be able to help add more. Capitalism does many things well, but making things safe for humans is not one of them.