Under the Hood of Luminar's Long-Reach Lidar

Luminar's lidar beams will be absorbed by water rather than scattered so fog will effectively seem black to it rather than white and it will be just as unable to see through rain or fog as normal lidar or your eye.

Thankfully if it's going to be developed in SF they'll be getting a lot of experience with fog.

Are there any cumulative effects regarding eye safety?

As in: if 100's of cars in your field of view, say a traffic jam in the opposing lane are all firing LIDAR would it add up to trouble?

I wonder how this affects non-human eyes.

In my understanding: Your brain does two things automatically that are currently difficult for computer vision: fast and good object recognition; and comparing the history of object to track it.

Currently CV can be good or fast, but not really both.

Also, contrariwise, your brain is actually terrible at depth perception and object recognition, and employs many tricks to approximate depth perception and recognition[0]

---

I don't have any special knowledge on this, but I wouldn't be surprised if self-driving car companies are using Lidar to train CV algorithms.

---

0. http://www.opticalillusionsportal.com/wp-content/uploads/201...

You have good company with that opinion. No less than Sebastian Thrun will tell anyone who will listen that he is sorry he got the self-driving car world hooked on Lidar point clouds. The Udacity self-driving car class is largely CV based. Cheap cameras and good CV will win eventually IMHO.

Also, our eyes are astonishingly good cameras - high dynamic range, excellent white balance control, high speed autofocus, high bit depth.

The human eye has much higher sensitivity range and resolution than any sensor I know of.

The human eye does the equivalent of what you might call supersampling in computer terms. The eyes make very subtle focus shifts to change perception of the same object and get a more detailed picture of something that is difficult to see. No mechanical lens that I know of can move this fast or accurately but you can achieve something similar with multiple cameras.

The brain also filters out extraneous data and interpolates missing data.

Human vision is far from perfect though. Light changes that exceed you sensitivity range will blind you. Vision is impaired by darkness.

The filtering and interpolation mechanisms of the brain are error prone and the brain often does not know when it is wrong. You may see something and never know it wasn't real, or don't see something and never know it was there.

This comes up basically everytime the Tesla Autopilot is discussed on HN since they rely mostly on cameras, except that they also have a front radar. In the end, we want self-driving cars to be better than human drivers, so they should also handle situations that humans are struggling with and additionally have some redundancy. Some things which are difficult to do for CV, detecting things in the dark for example, are trivial for Lidar. Radar on the other hand can provide accurate measurements for distance and speed of other vehicles, also independent of lighting conditions.

Yes. The human eye is attached to the human brain, which we haven't yet been able to reproduce for the machine.

Brains like yours are more advanced than the computers they are hooking these up to, and those brains still cause a lot of fatalities every year.

Why make the problem harder than it needs to be? Why not shoot for superhuman? Who cares if the submarine swims like a fish?

Lot of good comments here - CV is heading in that direction, but in addition to the other mentioned problems, the needed data bandwidth rises exponentially - my own take is that we go broad in some ML work where we should go vertical (I believe that the same thing that makes you jump when you see a (non) snake is what your brain is using for image fusion feature detection. However, the real problem not mentioned is parallax - we understand some bits of it, but our brain does a lot of high level manipulation of the data - in the real world, it's two completely separate incidence angles for 2+ visual sensors seeing a point at a given angular relationship, but the same point is not the same thing - take an index card, paint one side red and the other green, and look at it edge on far enough away both eyes see each side, and its close enough for parallax - sometimes produces interesting illusions :-) When you get to complex 3D structures and parallax, the problem gets a lot nastier, because after all the work to get the data you want, you are then forced to throw offending bits of it away and start making educated guesses - my own take is "You do not see with your eyes, you see with your mind" :-)

I think a simpler answer to the others given is because we can, and don't think it'll make the cars less safe.

We add ultrasound range sensors for parking because it helps, even though it's possible to park despite not being bats. You also could still drive with reduced vision, but it'd be better if you didn't.

Would we drive better if we had accurate distance sensors?

So we theoretically may not need to, but they help with the current problems we have.

Main use of lidar is for very fast obstacle detection. Other popular methods to detect obstacles are ultrasonics and vision.

Problem with ultrasonics is latency, range as well as lack of accuracy.

To detect obstacles with vision, generally you need stereo camera and generate disparity map (you can do with moving mono camera too but its much harder and even more unreliable). The confidence levels drops quickly as objects are farther and lighting conditions, reflections etc gets more complex. The main problem is that popular algorithms only rely on geometry to compute the depth and they don't into account knowledge of the world. For example, if there is area of specular reflection, geometry will fail even though one can reasonably estimate depth by assuming some continuity of shape of objects in real world.

We humans are very efficient in figuring out depth even with single eye and its purely because we can do very quick and accurate object segmentation and combine it with our knowledge of what those objects are (semantics) as well as our internal map of the world around us. For example, when you see truck on the road even with single eye, you have fair estimate of what sizes trucks usually are and how big should it appear if it was 100ft away vs 1000ft away. You would be able to do this even if you can see truck only partially from the side angle or back angle.

There is no known algorithms currently that can come anywhere close to human level accuracy and speed for depth estimation and consequently obstacle detection using just vision in variety of lighting conditions when you are moving at 70 miles/hr. On the other hand, lidars are super easy to implement, fast and very accurate. So unless a major breakthrough comes along, lidars are actually the only choice right now for fast and accurate obstacle detection. Remember even a tiny error rate could be fatal when you have millions of cars covering millions of miles every day.

Lidar works at night?

Passive lidar/radar/sonar is a harder problem than active.

Car-to-car communications are not necessary for automatic driving. They may even be undesirable, as a potential source of bad info. Google/Waymo doesn't need them. They make the point that there are lots of things in the environment, from bicyclists to moose, that aren't going to cooperate with the system.

The need for super-detailed maps is probably temporary. A reasonable goal is to have the ability to drive anything with a basic map (Google StreetView level) at slow speed, then retain info to do it again at faster speed. Some of that mapping info may later be uploaded for use by others.

A "digital rail" with centralized traffic control is much safer and much more practical.

Autonomous is a dead end. People make the mistake of looking at early progress (starting at 0 base line) and thinking it will not stop till it is perfect.

It is like when we went from 100mhz to 1Ghz chips and people were saying "in 10 years we will have 10Ghz chips" but in reality we never will.

I would settle for a car that drives exactly as well as I do, and non of that stuff is required to meet that goal.