Hacker News Reader

Comment by syntaxing | Hacker News Reader

Disclaimer, I work in the autonomous vehicle industry but the opinion are of my own. I roll my eyes (heh) when people say if humans drivers only use their eyes so AVs should too. What a load of bull. While a camera is getting really good, but HDR is still subpar to human range. What about stereo vision and how we use two eyes. What about how your eyeball can articulate while having stereo vision. What about how your neck also articulates your vision range. What about how your eyes instinctively look at moving objects. There’s so much more to “human vision” than just the eyeball.

View on HN

Everything you mention seems possible to replicate with 2 cameras, a few sensors (and a X years of AI/software dev). Which is the point of people saying cameras are good enough if human’s eyes are good enough.

Or am I missing something?

Two cameras, plus as-of-yet unknown human vision to synthesize depth info correctly for arbitrary scenes, plus the ability to articulate side by side by 8-10 inches, plus a very efficient liquid coating plus cleaning mechanism (blinking), plus the ability to deploy anti-glare shades proactively (hands).

Don’t forget the self maintaining general intelligence with a minimum of 14-16 years of real world training (minimum, varies by state) to understand context and environmental factors, go to highly trained specialists when sensors seem out of calibration, etc.

So that’s the good side. Now add in the bad sides. Sometimes drunk / high. Often tired. Night blindness as you get old. Maybe cataracts.

Beating the prime of the prime human driver with cameras may be hard. Beating tired / drunk / old drivers is a different story.

Depends on how buggy the firmware is and how poorly maintained the sensors are I’m guessing?

>plus as-of-yet unknown human vision to synthesize depth info correctly for arbitrary scenes

I have an adversarial example.

https://wallpapercave.com/wp/InAcQKW.jpg

I think he's saying that the "X years of AI/software dev" are really the important part, much more than the mechanics of the visual sensing. And that the human brain and optical system doing that is what's not so easy to replicate in a machine.

2 cameras with 3-axis rotation and xyz movement to look around obstructions and see things out of reach from a fixed perspective.

Systems wise, that's going to be a lot more complex than having more cameras. Of course, if you've got enough cameras you can probably get better visibility than a human, and continuously process all viewpoints. But 2 cameras is definitely insufficient.

The human eye is a far better sensor in many ways than the CMOS sensors used in cars. They have variable focus lenses, finely controlled irises, and are on gimbaled mounts in the head. They have integrated shades/shutters and cleaning mechanisms. The head itself is on a gimbaled mount which itself can be moved. Attached to the eye mount are inertial sensors, audio sensors, and an accelerometer.

A camera or even a stereo pair of cameras mounted in or on the car will provide inferior imagery to the control system than eyes to the brain. They have less dynamic range and no articulation. If you wanted to replicate human style vision you'd need a bunch of fixed cameras and inertial and acceleration sensors all on top of AI that's better than what Tesla's been demonstrating.

LIDAR is the most straightforward augmentation for fixed cameras because it can build very accurate depth maps and image segmentation. You need fewer fixed cameras if your spacial model is built with LIDAR. You're in even better shape if those systems are augmented with radar.

While humans don't have LIDAR and such, our visual systems are highly developed and augmented with highly developed proprioception. Trying to replicate it with just cameras and tons of processing power is a fool's errand.

First worked on Autonomous cars in 2007 for DARPA grand challenge and even back then fused sensing was where it was at. Modern Cameras are better but they're no replacement for the eye. The best thing we can do right now is take high quality cameras and augment them with things like radar and LIDAR to get close to the human eye level perception. It makes the AI's job more about the macro driving problems and less about vision. Look at the string of crashes of Teslas into white box trucks on bright days.

I remember the first time we had problems with a matte black surface with our LIDAR that would have been easily spotted by our camera and vice versa with a shiny white surface in direct sunlight relative the car being easily picked up by lidar but nearly invisible to the cameras.

It seems insane that Tesla continues to shun LIDAR. Is it just pride at this point? Apple sells an $800 tablet with it built in, so while obviously the Tesla unit would have to be a lot larger and more powerful, I can’t see the cost argument making sense when the cheapest car they sell costs 50x as much (and the most expensive will be 250x).

I think at this point Tesla's camera-only system is an issue of pride for Elon Musk more than anything. He makes declarations and then refuses to backtrack on them ever unless he is literally forced to do so.

In theory a camera-only autonomous system can work effectively but in practice there's innumerable edge cases where it doesn't work well and edge cases are where catastrophic failures happen. If you had infinite processing power and error free AI you might be able to cover many edge cases but Teslas have neither.

Worked on autonomous X for US gov. For any value of X, its a fool's errand to try to reduce the amount of information going into your GNC. Multi spectral, multi viewpoint, inertial-calibrated, and lots and lots of onboard models and processing.

The best cameras are about 80% there, compared to human eyes. At least, in some respects. There are no cameras that are 80% there in all respects. And that’s when compared to low-quality human vision that is attached to low-attention/highly distracted humans.

Now, there’s this little thing called The Pareto Principle.

That last 20% is going to take a long time to achieve, and is going to be very, very expensive.

Are you willing to roll a ten sided die every time you get in the car, and only if you roll a three or higher, do you get to arrive at your destination unhurt, on time, and without major incident?

Plus, you don’t only use your eyes. Your ears sense sound, the balance-sensing thingies in your ears, even the vibration sensed by your butt feed into your brain’s awareness of what’s going on.

The human eye has incredible dynamic range -- it's something like 25-30 stops total (although you can't see them all at once). Compare that to the 10-12 stops for a high quality DSLR and the 5-7 for the types of cameras going into cars.

The dynamic range is the kicker. Automotive cameras are already grayscale to get as much light as possible and not compromise vision at night but its still far inferior to the human eye (and even that is bad! but it's the benchmark Elon chooses).

No surprise then that the vision-only "level 5 autonomous driving" system would want the high beams on...

That's quite one-sided list which imo doesn't really prove anything about whether human eyes or cameras have an overall advantage. We could make a similarly one-sided list in favor of cameras: E.g. one could cite faster adaption rates of cameras, 360° vision, ability to capture infrared photons, possibility of wider stereo separation etc.

We don't really use stereo vision for the distances required for driving. Our eyes are simply too close to each other for that.

Stereo vision is mostly usable below 15 ft/5m range, if even that.

While I think self-driving will be just fine with cameras, human stereo vision works at much longer distances than 15ft: you can easily test it by looking at few distant trees (150ft), and then closing one eye with a hand and blinking. You’ll see that there is good Z-layer separation with two eyes and none with one eye.

A question I always wanted to ask: why stereo vision cannot replace lidar?

It's just less accurate, more error prone, and more limited. Especially in bad conditions and when there is low contrast.

Stereo vision can also be fooled by things like reflections.

Stereo vision at the human level also requires head movements and so on.

Also people don't realize how important blinking is. Any camera system that is not thoroughly self cleaning is going to have significant problems after a relatively short period of time. We don't have cameras built to withstand the elements day-after-day like our eyeballs.

Doesn't lidar have the same problem then?

What about how birds use wings to fly?

And what about how fish use fins to swim?

Reply on news.ycombinator.com