Lidar Is a Crutch
lidarmag.com
lidarmag.com
It seems like the teams working on autonomous vehicles need all the crutches they can get, and it seems like a good idea to lean on superhuman sensors to make up for subhuman "cognition."
I do wish, though, that people would stop equating all radar with the adaptive cruise control style radar - imaging radars are a thing that exist, and can be competitive with lidar.
If I had 360° night vision this wouldn't have happened. I can't trust my own two eyes. I wouldn't trust anyone's two eyes for that matter to drive a car safely. They weren't good enough last year and they won't be enough next year. Replicating humans will just replicate their weaknesses. I'll take all the crutches I can get.
By giving a car more than two eyes it has the potential to become a better driver than me because it can watch more. By adding more than just visible light cameras it can see things that I would never see. I want my next car to be safer, the big limit is not seat belts, brakes, crumple zones, airbags, and such - the big limit is the limited human in control. There is potential to make some of the above obsolete if the control system is enough better.
This isn't really the point, it's that humans are able to successfully build an accurate-enough model of the world to navigate terrain with only camera like vision -- and a really bad one at that.
There is, of course, no problem with giving cars more sensory abilities to make them safer but what they mean by a crutch is that the CS breakthrough that will make self-driving cars compete with humans will function about as well with two crappy cameras mounted on the drivers seat as it would with an array of high-precision sensors.
The idea that you can throw a cheap camera in a car and think you can achieve the same always seemed strange to me.
Unfortunately resolution of your eyes drops off quickly from the center of vision. Yes you can move your eyes to focus on different things, but so can cameras.
Sure the mental image you build is high resolution, but not that directly related to reality. A good example of this is the numerous optical illusions that depend on you looking at a point of an image, building a mental representation of that image, then finding a conflict whenever you move your eye.
Link that seems to be the source of the 576MP number: https://art-sheep.com/the-resolution-of-the-human-eye-is-576...
Not what I'd call a particularly scientific explanation.
True, but we can move our eyes around very quickly. Cameras in self-driving cars are usually fixed.
Also, human eyes have better dynamic range.
Let's not forget about human experience. It gives context to any visual perception, and can't be replicated by cars. We simply have access to a larger, more complex environment, and the environment fosters our intelligence. It's easy to overemphasise the brain to the detriment of environment, but in reality the environment is the main cause for the development of intelligence in the brain. The corollary is also true - a lower complexity environment will lead to lesser intelligence, no matter how powerful the brain is. Intelligence springs where agent needs meet with external limitations, those limitations guide its development.
By analogy, the environment is like the training set and the brain is like a deep neural net. A deep neural net with a poor training set is not going to be accurate. That's why DeepMind, OpenAI and others have focused so much on artificial environments (games) to train agents - they know that environments are the key to training advanced AI. A mere dataset is like a static environment where nothing new happens. Research is transitioning from datasets to simulated environments for the next step in AI. I'd even go as far as to track the evolution of AI by the evolution of environments and environment models.
As for the point about the training environment, a lot of what AI cars do is learn from simulations.
unless they wrote it down.
You can read all the books and notes about WW1 battles, you'll never come close to feel what they felt in the trenches.
Even if that's true, I (and almost anyone) can also drive cars in computer games at 1080p or less.
This is a spectacularly misleading claim. Humans have a very small field of high-definition vision, and a wide range of low definition vision, easily exceeded by almost any camera system. Where humans excel is in the heuristics of taking marginal information and teasing out meaning (not always the correct meaning), though neural networks approximate that system.
Tesla has something like 10 cameras running 1280x960 @ fps or similar. With so few data points (and no color) you can't identify similar vehicles. After all snow plows, school busses, ambulances, bikes, motorcycles, police, etc all have different behaviors. Some even signal cars with spot lights, police red/blue lights, brake lights, blinking high beams, etc. All would be invisible to lidar and contribute to lidar controlled cars acting less like human driven cars.
Biology got to be just good enough to make it to tomorrow. Photosynthesis is one of the oldest mechanisms around right? Quick google and at the moment that turns sunlight to energy tops out at 6%, now look at solar panels https://upload.wikimedia.org/wikipedia/commons/5/5d/PVeff%28...
Now, commenting on biology and electronics are topics vastly outside of my ability. I'm just trying to make the point that we can't be the best it can get, surely? We long left the time where industrial progress meaning "pick one things and do it well"
It's all about the data processing and the breakthrough that will enable human-ish self-driving cars will be there and not in more accurate sensors.
yep, resulting rich details simplify stereo matching a lot. Instead of complicated algorithm it allows our brain to just run simple displacement match in extremely parallel fashion natural to the brain.
>The idea that you can throw a cheap camera in a car and think you can achieve the same always seemed strange to me.
it isn't strange, it is just a bit early. Running 1MP stereo on a single core P4 in 2004 was just pitiful. These days a 20MP sensor costs under $30 and 16-32 cores is having much better time with it (using GPU is much cheaper and several times faster - it is just that i have these workstations around with good CPUs and no GPU to speak of. Tesla for example does run powerful GPU for their cameras.) So, scroll forward another 15 years, and we'll have nice stereo with something like 200MP sensors. I honestly don't see how lidar can do even just 4MP at minimally usable 30FPS in any foreseeable future (as probing at 150m is 1 microsecond per pixel).
However, the focal point with respect to the sensor is indeed miscalibrated in some units, and good units can become miscalibrated over time. There are some after-market correction devices available.
:)
At least as important, and something that comes up shockingly little as well (I don't see any responses to you yet on it here even!), is that overall humans stink at driving. Like you said, arguments begin with "because humans can do it" and need to be stopped right there, because that needs a lot of qualification. "Humans can do it"... with well over 1.25 million deaths a year worldwide [1] and another 20-50 million injured/disabled. Also these stats are actually worse then they look at first glance, because it's not a uniform age distribution. A lot of young people die on the roads vs other causes of death, which represents an even bigger loss in terms of human years.
Personalized arbitrary point to point mechanized transportation is so insanely valuable that it's well worth that kind of death/injury rate, or for that matter one vastly higher (as it was in previous decades before modern safety standards), but there is also no reason to use it as a benchmark for "acceptable" if the opportunity exists to do better. And in particular there is no reason to limit autonomous systems to human visual spectrum for information input. That doesn't cut it for a lot of adverse weather conditions, or even just for humans along the road or animals jumping across the road at night for that matter. A big part of the pitch shouldn't be to match humans, it should be to exceed them and by a sizable margin as well.
----
1: https://www.washingtonpost.com/transportation/2018/12/07/tra...
Some people claim that they won't make the same mistake twice, but how many times have you seen the same bug crop up after it was fixed in previous updates?
I would also posit that the existing statistics provide a "good enough" floor. If a million people currently die, the software will be good enough when it has the same level of fatalities. There is no incentive to make it better, at least not in our Capitalistic system. Investments would be better spent on differentiating widgets that dazzle the occupants, not protecting non-customers outside the vehicle.
This seems like a fallacious argument from incredulity. There is no reason to believe that humans are the ultimate existence for navigating around multi-ton land vehicles at 20-100+ miles an hour, and there is plenty of reason not to. For computers, there will be bugs and edge cases, but there are plenty of examples of high assurance low bug code in areas it matters. Additionally, any experience gained can be applied to the entire fleet at once. Computers have fundamentally better response times. And sensor input absolutely can better. A self-driving vehicle need not have any blind spots in 360°, and can "watch" all of that at once. No human will ever handle animals at night the way a vehicle with a FLIR could.
Meanwhile we know for sure that there are whole classes of issues that simply do not exist for computers. They do not get tired. They do not use drugs. They do not get distracted by the passenger or texting (humans are awful at multi-tasking). They do not "have a bad day". They do not get angry at that "jerk" in the other lane. They will know all the rules of the and not forget. Perhaps most of importantly of all, for all these things there will not be variability within a fleet. Sure, there are very good human drivers, but there are also endless awful ones, and every single person must be trained anew and go through a high risk phase while they get experience. Not with a computer. Consistency and attention to boring details and checklists, every single time, has proven a key driver of health & safety improvements across many industries. And even all that are just a few of the first order effects, there will be second order ones too like much less incentive to put off maintenance when the car can drive itself to the service center and another can take its place.
Sure there will no doubt be more obstacles along the way and it'll take a lot of time and effort. There may well be high profile screw ups, it's still humans doing the developing. But you've got a real steep hill in my opinion if you can look at all that and the casualty stats and still want to argue that computers won't ultimately be better.
>There is no incentive to make it better, at least not in our Capitalistic system
This is just utter bullshit though. Our capitalist system includes cost internalization in many cases, and cars are one of them. It's called insurance, and since you're required to carry it by law the fact that it didn't occur to you raises will questions in my mind about the basis of your opinions. Unlike many fields, in cars improvements in safety have a direct financial bottomline. Furthermore, self-driving will most likely enhance that, not detract it, because there the model of people giving up direct ownership and buying shares instead will become possible.
You also say lidar isn't a crutch because self-driving cars aren't ever really going to be able to go by image recognition and enough logic to know what to do - rather, they're going to go by the complete plotting of the traffic grid with lidar being there for object-avoidance. This is Google's model and they are furthest along - and set-up in Arizona where the unpredictable is kept to the minimum.
Indeed, and that's how autopilot driven buses, that were around for at least a decade, work. Except their movement trajectories are predetermined, and they are very slow, and stop at every obstacle imaginable.
I see that argument a lot, "well humans only have two eyes so of course two cameras would be enough".
Well monkeys have two eyes too but they wouldn't be driving on the autobahn anytime soon would they.
That argument is really used by people who have no idea how any of this works, and it really is frustrating.
Notably however, the radars employed by current Tesla cars lack the angular resolution to distinguish between stationary objects next to the road, and stationary objects in the middle of the road.
This is why Teslas keep running into stationary cars. The radar in them is not designed for this, and can't tell the difference between a fire truck in the lane and the barrier to the side of the lane.
But there are a lot of possibilities between zero autonomy and level 5 autonomy that might almost require lidar too in the short term.
Is there a better way to build a better autonomous car? We're at the initial stage of exciting people with an implementation that kind of works. We will now spend generations selling better and better cars with different, fewer, better, cheaper sensors and other parts.
I don't think Edison (and his employees) got a momentary flash of light from a lightbulb and then said, "alright, it's almost working, now let's see if we can do it without the fragile glass bulb."
Effective incandescent light bulbs were also around before Edison's. The main things Edison's team brought to the table were a better filament material, a better vacuum in the bulb, and supporting electrical infrastructure (generators, etc.)
In a perfect world, yes. But cost always comes into the equation. Companies like comma.ai and Nexar are getting their devices in to the hands of hundreds of thousands of people at this point. When you can get that level of training and autonomy from a simple camera based device it becomes a really compelling product, even though you're nowhere near full level 4 self driving.
Edit:
And neither is Tesla. Anybody operating either of these systems without paying attention to the road and being prepared to control the wheel at all times should have their drivers license confiscated.
See page 73 of the Model 3 owners manual: https://www.tesla.com/content/dam/tesla/Ownership/Own/Model%...
Warning: Autosteer is a hands-on feature.
You must keep your hands on the steering wheel at all times.
Warning: Autosteer is intended for use only on highways and limited-
access roads with a fully attentive driver. When using
Autosteer, hold the steering wheel and be mindful of road
conditions and surrounding traffic.*
Page 74 makes this even more clear:
Warning: Autosteer is not designed to, and will not, steer Model 3
around objects partially or completely in the driving
lane. Always watch the road in front of you and stay prepared
to take appropriate action. It is the driver's responsibility
to be in control of Model 3 at all times.Until it doesn't.
Failure to stop for a stationary gore point:
https://arstechnica.com/cars/2018/06/ntsb-autopilot-steered-...
https://abc7news.com/automotive/i-team-exclusive-tesla-crash...
Failure to stop for a stationary fire truck:
https://www.wired.com/story/tesla-autopilot-why-crash-radar/
Again, the Model 3 user manual is illuminating:
Warning: Traffic-Aware Cruise Control cannot detect all objects and,
especially in situations when you are driving over 50 mph
(80 km/h), may not brake/decelerate when a vehicle or object
is only partially in the driving lane or when a vehicle you
are following moves out of your driving path and a
stationary or slow-moving vehicle or object is in front of
you. Always pay attention to the road ahead and stay
prepared to take immediate corrective action. Depending on
Traffic-Aware Cruise Control to avoid a collision can result
in serious injury or death. In addition, Traffic-Aware
Cruise Control may react to vehicles or objects that either
do not exist or are not in the lane of travel, causing Model
3 to slowdown unnecessarily or inappropriately.
Warning: Several factors can affect the performance of Automatic
Emergency Braking, causing either no braking or inappropriate
or untimely braking, such as when a vehicle is partially in
the path of travel or there is road debris. It is the
driver’s responsibility to drive safely and remain in control
of the vehicle at all times. Never depend on Automatic
Emergency Braking to avoid or reduce the impact of a collision.
Warning: Autosteer is a hands-on feature. You must keep your hands on the steering wheel at all times.
Warning: Autosteer is intended for use only on highways and limited-access roads with a fully attentive driver. When using Autosteer, hold the steering wheel and be mindful of road conditions and surrounding traffic.
Warning: Autosteer is not designed to, and will not, steer Model 3 around objects partially or completely in the driving lane. Always watch the road in front of you and stay prepared to take appropriate action. It is the driver's responsibility to be in control of Model 3 at all times.
Warning: Traffic-Aware Cruise Control cannot detect all objects and, especially in situations when you are driving over 50 mph (80 km/h), may not brake/decelerate when a vehicle or object is only partially in the driving lane or when a vehicle you are following moves out of your driving path and a stationary or slow-moving vehicle or object is in front of you. Always pay attention to the road ahead and stay prepared to take immediate corrective action. Depending on Traffic-Aware Cruise Control to avoid a collision can result in serious injury or death. In addition, Traffic-Aware Cruise Control may react to vehicles or objects that either do not exist or are not in the lane of travel, causing Model 3 to slowdown unnecessarily or inappropriately.
Warning: Several factors can affect the performance of Automatic Emergency Braking, causing either no braking or inappropriate or untimely braking, such as when a vehicle is partially in the path of travel or there is road debris. It is the driver’s responsibility to drive safely and remain in control of the vehicle at all times. Never depend on Automatic Emergency Braking to avoid or reduce the impact of a collision.
For example, assuming 3 lidar units on the front of each car, in a large intersection, say two lanes of cars turning in front of 4 lanes of cars pointed at them, waiting at a red light. That would be hundreds of potential reflection/collisions per second.
The user manual for an older Hokuyo lidar with a single scanning plane (for example, mounted on a forklift to avoid hitting a wall) mentions to offset the mounting/angle slightly across the fleet to avoid interference.
More specifically, each detector is looking through a soda straw at a tiny spot in space, and rejecting anything that doesn't arrive within a narrow range of time, which also has to have the right light frequency and something like a pseudorandom code sequence which gets match filtered against itself.
In practice, you'd have to send the right wavelength of light to the right 1cm circle in space, send it during just the right few hundred nanoseconds, send it with the right pulse sequence (which one could determine but would be unlikely to happen by chance). Even if it happens once, it would have to happen many times at the imaging rate of the sensor to cause major issues...
[1] https://books.google.com/books/about/Radar_Handbook.html?id=...
Think of it as encoding your mac address on each packet you transmit wirelessly so you can tell which signals come from which source.
In the case of Automotive radar, using it as an identifier to prevent interference is appealing because it allows each radar to use the full 6 GHz of channel bandwidth which greatly improves the performance of the radar despite the fact that there are tons of same channel independent devices operating. Each one would effectively be a jammer otherwise.
As I watch Tesla's "Autopilot" do its thing, it seems its limitations are less about object recognition and more about what do with the data it has. Lidar won't help it intelligently handle a merge of two lanes into one, or seeing a turn signal and slowing down to let the person in, or seeing a red light and knowing to start slowing down before it recognizes the car in front of it slowing down, or knowing exactly what lane to enter on the other side of a tricky intersection without a car in front of it, or knowing that car X is actually turning right so you shouldn't start following it if you're going straight, or having the car see a clear path ahead and having it accelerate to the speed limit when all other cars are stopped or going much slower in adjacent lanes, or moving to the left to allow a motorcycle to lane split, etc., etc. It's still great, but there are a ton of things to learn.
Maybe once there is no human in the driver's seat, you'll need the extra precision that lidar provide, but there are big gaps before even we get there.
A lidar gives you a stream of 3d points in the order of megavertices a second.
The processing pipeline for any visual system is at least frame rate of the camera (the faster the camera, the less light it can get) plus the GPU transfer time(if you are using AI) then processing.
this means you are looking at ~200ms latency before you know where things are.
Lidar is a brilliant sensor. Maybe it will be supplanted by some sexy beamforming Thz radar, but not in the near future.
you are limited by shutter speed, as you know, even if the shutter isn't global. (lidar is too, but we'll get to that in a bit)
Any kind of object/slam/3d recovery system will almost certianly use descriptors. Something like orb/surf/sift/other require a whole bunch of pixels before they can actually work.
only once you have feature detection can you extract 3d information. (either in stereo or monocular)
A datum from a lidar has intrinsic value, as its a 3d point. Much more importantly it does drift, event the best slam drifts horribly.
Lidar will be superior for at least 5 years, with major investment possibly 10.
It certainly seems like it's still needed. Some of the high profile Tesla crashes seem to be caused by the camera system not properly recognizing the side/back of a truck or the divider on an off ramp and plowing right into them where lidar probably would have.
The terahertz radar people are slowly making progress. Good resolution, almost as good as LIDAR. Better behavior in rain. Beam steering with no moving parts. Plus you can tell who's carrying a gun.
Musk is in trouble. No way is Tesla close to real automatic driving. Does Tesla even have reliable detection of stationary obstacles in a lane yet? Their track record on partial lane obstructions is terrible. So far, Teslas on autopilot have driven into a street sweeper, a stalled van, a fire truck, a crossing semitrailer, a lane divider....
> Musk and almost everyone else in the business recognize self-driving cars and trucks as the future of automotive transportation.
Those people in their prime majority think that some kind of "AI" program will be somehow think over the inputs and tell where to drive. They all have too much expectations for technology coming under the word "AI" these days
They all do so without understanding what those "AIs" actually are, and not realising that such programs can't make any "cognition."
Until that change, any talk of "self-driving" is premature. This does not preclude however the coming improvements in cruise control, and computerised collision avoidance.
He has no substantial CS or technical background. He dropped out of his physics masters.
Give him an evaluation.
Also, he has a startup on AI: https://en.m.wikipedia.org/wiki/OpenAI even if you think he's not proficient in technical aspects of AI, he definitely has a lot of ideas about social implications of AI.
He left in 1992 to study business and physics at the University of Pennsylvania, and graduated with an undergraduate degree in economics and stayed for a second bachelor's degree in physics. After leaving Penn, Elon Musk headed to Stanford University in California to pursue a PhD in energy physics.
This is total rubbish. I don't believe for a second that taking a self-driving uber to work and back everyday will be cheaper than driving my own car.
Brad Templeton did some math and thinks that self driving cars will be cheaper on a per-mile basis if you sometimes take smaller cars when you need to.
There would be some markup so the owning company has a profit margin, but the higher the price is, the greater the incentive for competitors to exist, which would exert downward pressure. So maybe you'd end up paying 1/20 the price of owning a car.
If you don't agree, why not?
If I used Uber every time I’ve used her corolla in the past 10 years it would far outweigh the cost of gas, insurance, and fixing the car.
Uber pool is cheaper but why would I inconvenience myself so much for a 2 miles radius.
When people learn they have to pay for cleaning/repairs if they don't take care of the car then they will be much more careful keeping it clean.
Even in the best case, there are lot more people who want to get around during "rush hour" than the rest of the day, so most shared cars could at most work for 2-3 people each day. The types of people who are on the road during not rush hour are more likely to have a larger group (kids) with them so the shared car needed for them is a different size.
I don’t think a subscription model is too far fetched at all. We definitely intend to at least consider being a one car household, even at current ride share prices, when the current secondary car reaches end of life.
I do live in Texas though, so my experience might not be typical.
If you live in the city:
- Cost of Ubering to/from work could be about $3.50 per trip, or about $1,750 per year.
- TCO of car ownership (based on this tool[1]) is $878 per month, and with $125/month for parking (a lowball), you get $12,036 per year.
If you live in the suburbs:
- Uber estimate one way from Phoenixville to downtown Philadelphia is $43.41 with Uber Pool. That's $21,705 per year.
So if you live in the city, owning a car would cost you 6.87x more money. But if you live in the suburbs, Uber would cost you nearly double that.
[1] https://www.nerdwallet.com/blog/loans/total-cost-owning-car/
(Keep in mind if you already paid off your car, your yearly TCO is still $4,260 per year, or 2.43x more than using Uber if living in the city)
I'm keeping my old car while it's viable, but after it stops being, I'm not sure what I'll do (there is some convenience to add there too).
A word on that software: As it currently stands, they are delivering software that enables Level 2 capabilities. This is sometimes called "hands on", as in your hands should remain on the steering wheel and your eyes on the road, ready to take control in an instant. According to Tesla, drivers are to keep their hands on the wheel and pay attention to the road; if the driver fails to do so then they are at fault for any accident. However Elon Musk contradicts this company policy and has promoted the system as hands off on national television. Why would he do something so irresponsible? Because misrepresenting the hardware and software capabilities of his cars helps him sell cars. He knows the hype for self driving cars is at a fever pitch, and stretching the truth helps him profit from that hype.
https://www.businessinsider.com/elon-musk-breaks-tesla-autop...
Now consider MobilEye, a subsidiary of Intel and a major player in the field of camera/radar driver assist technology. Tesla was using MobilEye tech, until MobilEye terminated that relationship because they believed Tesla was being irresponsible with how far they were pushing MobilEye's tech. MobilEye had a financial incentive to see Tesla succeed with a camera/radar only solution, and continues to have a financial incentive to downplay the necessity of LIDAR. But do they? No. Instead you've got MobilEye's CEO Amnon Shashua talking about the virtues of a combined LIDAR/radar and camera/radar solution, while trash talking Elon Musk for being irresponsible.
When you consider who is saying what and what their financial incentives are, it becomes clear that Amnon Shashua is an ethical person and Elon is a car salesman who is making technically unfalsifiable claims about the capabilities of the hardware in Tesla cars to profit from automation hype.
The point being Tesla has actually shipped a product that's pretty good, and therefore, is currently the best in the market. Sure maybe Waymo's system is better but since I can't use a Waymo I don't think we should be counting it just yet.
"Waymo One is only available to a small group of consumers who also participated in the early rider program."
[1] https://www.extremetech.com/extreme/281776-waymo-launches-co...
However with 1 billion miles driven on autopilot and (IMO) relatively few problems it seems like it must be pretty good. I hear about accidents involving uber, waymo, and similar just about as often as Tesla, despite Tesla shipping 5000 cars a week or more. Granted only a faction of those are likely to have the $5,000 enhanced autopilot, still radically more miles driven than anyone else.
Sure, watching a Disney movie while a Telsa drives is stupid and got someone killed. Seems like a pretty poor proof that lidar is superior. Not to mention Tesla hardware and software has some pretty far since then.
However, the car was programmed not to brake...
The problem was the software, not the hardware.
Sure Lidar + camera would be ideal, but if picking one or another it's far from clear where would be safer.
Lidar has 3D of course, but a MUCH lower data rate, and of course no color.
Believe it or not, cars will need to be able to drive in the rain, so if you're going to eventually need to build a car that will work reliably without lidar then why not build that to begin with?
It looks like most SDC companies are going with lidar because it is faster and easier, but if it only covers 90% of use cases then that does sound like a "crutch".
Ducttape is a crutch-crutch when applied correctly.
As far as I can tell Musk insists walking unassisted with a broken leg is better than using a crutch because a good leg is better than a crutch. Sure, but he's not providing a good leg, he's providing a broken one. It's like telling people to drag their broken leg today because 3 months from now it will be better than a crutch. And he's not taking this route because today's cars do better with normal cameras than with LIDAR (future cars might though what good is that today?) but because it's cheaper while still allowing big claims.
Use the right tool at the right time. In the meantime develop the next tool and start using it when it becomes appropriate.
The tesla system has 10 cameras, but I believe only two of them look forward. I believe they are 1280x960 @ 30 FPS or 36M pixels/sec. But there's two of them, 73M pixels/sec. Each pixel is in color (lidar is just a distance).
So the tesla system has WAY more information about the environment, granted distance has to be inferred, but it also had radar to help with that.
Additionally being inherently more similar to eye sight, a car using cameras is likely to get along with human drivers better. Slowing down when it's foggy or rainy, seeing at similar ranges to humans, and being able to use color for additional context. Is that a UPS truck or an ambulance? Is that a reflection of a police car with it's light on or just a window reflection? Is that a boulder or a wind filled trash bag?
I'd argue the contrary. Intelligence is primarily not about the amount of data, but the amount _and_ quality of data you receive. If I would have a magic sensor giving me obstacles in a segmented form, that would be couple of KB, and it would beat any other sensor on the market.
Inferring the distance from stereo images has its own failure-modes and are not easy to account for as in LiDAR. LiDAR also gives you reflectivity, so you will be able to differentiate between a UPS truck and an ambulance.
> Is that a reflection of a police car with its light on or just a window reflection?
Fun thing, to my knowledge reflections are a major unsolved problem for vision. It is easier for LiDAR, as you can rely on the distance measurement and will have an object somewhere outside of any reasonable position (e.g underground, behind a wall). Depending on the lidar, the glass might even register as a second (actually primary) return.
Yes, you need cameras (likely color) to be able recognise any light based signalling (traffic lights, ambulance/police lights...), so LiDAR is not the panacea. But having the lidar telling you that there is a window and that police is behind it is likely vastly more robust with lidar.
Also, the difficulty is that you have to see arbitrary objects, on the road and possibly stop for them. As long it is larger than maybe a couple of centimeters (or an inch), it will show on the LiDAR, with stereo vision, you need a couple of pixel texture to infer it.
300,000 per second... if you are trying to figure out what's going on in 1/20th of a second that's only 15,000 points. Assuming your scanning 3 lanes (3.7M each) out to 100 meters, say 3 meters high that 3330 cubic meters. So lidar gives you 2 points per cubic meter. Not exactly going to be easy to tell a bicycle from a motorcycle, or an ambulance from a ups truck.
From what I can tell machine learning has led to near human levels of object identification, not nearly as competitive for things like sparse monochrome point clouds.
At 65 MPH, to be able to avoid something you need some lead time, which means distance. The lidar stuff I've seen is pretty sparse that far out. Of course the sexy almost real looking detailed landscapes from lidar are from tripod mounts and long sample times.
Which leads me to the relevant question. Do you have any reason to think that machine learning will handle lidar at 180 feet range (2 seconds at 65 mph) than a pair of color cameras running at 1280x960 @ 30 FPS?
I ask because, as I understand it, humans actually have quite poor visual acuity through most of our FOV, with a small very precise region near the center. the visual cortex does some nifty postprocessing to stitch together a detailed image, but it seems to me that human vision is mainly effective because we know what to pay attention to. when I'm driving, I'm not constantly swiveling my head 360 degrees and uniformly sampling my environment; instead, I'm looking mostly in the direction of travel, identifying objects of interest, and taking a closer look when I don't quite understand what something is.
is it possible for a lidar system to work this way? maybe start with a sparse pass of the whole environment at the start of a "cycle", and then more densely sample any objects of interest?
But! The lidar data is useless by itself since the car is moving through space at an unpredictable rate. Each sample has to be cross-referenced by timestamp with the best-estimate location of the car in order to be brought into a constant frame of reference. This location estimation is a complex problem (GPS and accelerometers get us most of the way there but aren't quite high-fidelity enough without software help) so it can't be done onboard the lidar.
So to do what you suggest, the lidar would need a control system that allows its operational parameters to be dynamically updated by the car. But what parameters? Since the laser is already pulsing at least hundreds of thousands of times per second, there's probably not much room for improvement there without driving up cost, and if we could go higher we'd just do that all the time anyway. The only other option would be to slow down the rotation of the unit while it sweeps over the field of view we've decided is interesting.
That way is a little more conceivable, but I doubt it would work out in practice. If the unit averages 10 rotations per second, it has to be subject to 20 acceleration/deceleration events, which would be a significant increase in wear and tear on the unit. It would also make it harder to reliably estimate the unit's rotation at any point in time, again driving up costs.
All this can't grant you much more than, say, a 100% increase in point density on the road (assuming 120 degrees of "interesting" FoV and a 1/6th sample rate on the uninteresting parts). If these things are to be produced at scale, I imagine it would be easier to increase point density by just buying a second lidar, which would also bring better coverage and some redundancy in case of failure.
Combining the real-time data together is straightforward, but figuring out how to optimally take advantage of it all is definitely a challenge. Having the extra dimension or sense definitely helps though.
I see it helping most in tasks that require a lot of dexterity, such as multiple digits working together, where a camera lens would be too close to the object or covered by the item in the task (folding a blanket etc), blocking light.
The main downside is cost.
A splat of dirt on the LADAR, and it will be not much better.
MM and Thz wave radars are cheaper and better for this application.
What would be the point of it other than distracting you from, you know, actual driving?
You want to add yet more information to that.
What I expect in the next couple of decades is that we'll have certain individual roads and neighborhoods where human-drivers are banned, and maybe certain roads and neighborhoods where AI-driven cars are banned. Most roads will allow either, because it's not practical in most cases to maintain two separate road networks.
Being able to drive a windowless car by head-mounted-display is something we could accomplish with maybe some minor tweaks to current technology. I doubt such a system would be safe or legal, though, because a software failure could be fatal and we're really bad at making complex software that is also reliable.
The problem I have with the article is he said Lidar will never come done in costs to that of a headlight.
Tech marches on and headlights are not really that cheap, in the future I would not be surprised with mass production that Lidar does get that cheap.
How much did the first Kim-1 computers cost? How does that compare to a simple Arduino with a hundred times the power and memory cost today?
The only sensor that can give mm accurate, high resolution, long range 3d spacial data with a low latency, is lidar.
For a purely visual system to supplant lidar we need:
1) a self cleaning all weather/all light condition camera with ~170 degree field of view at ~30 megapixels
2) a Structure from motion system that has a sub 20ms latency, and can work with spherical images, at full resolution.
3) a semantic object recognising system that is able to classify any object class. It must be scale, rotation, colour and occlusion invariant. It must also update the worl map generated by 2
4) An object threat management system, that can take semantic information from 3, rank it in order of threat, to be passed to 5
5) A world motion prediction engine, that takes threats from 4 and works out if said object is likely to collide with the car
all inside 70-100ms
Thats not going to happen soon. Of the whole list, 2 is the closest to working.
Lidar allows you to cut through 90% of this, because it gives you an accurate point cloud. From there you can do clustering to make objects, and track those clusters to measure threats. All of this is doable on a small CPU now. Without AI.
I thought lidar was quite sensitive to dust, rain, snow, and fog. This is particularly worrisome because lidar samples at a MUCH lower rate than a camera (from 30M samples per cheap camera to 300,000 or so for an expensive lidar).
While lidar is pretty impressive at short ranges, what about at 2 seconds away @ 65 MPH? Will it detect a deaccelerating car faster than a camera that can detect a brake light? Will it be better at detecting perpendicular cars vs parked cars at that range?
Will weather cause the lidar to decay in similar ways to human eyes?
65 MPH is about 30m/s, your average lidar should be good upto about 200 meters, which is about 6 seconds at 65 mph.
> Will it detect a de-accelerating car faster than a camera that can detect a brake light?
Now this is an interesting question. yes and no. A lidar will on its own will not give you object recognition. It will tell you that a reflective surface of size _n_ is directly in you proposed path, and that since last scan its got closer. From that you can make a very accurate obsticle map.
I don't think that running a car soely on Lidar is actually all that feasible or a clever idea. Not without a boat load of mapping and processing to create a machine readable semantic map first.
Having a camera array _as well as_ lidar is a very good idea, as it can provide blended information from the lidar to the semantic map being generated by the camera and radar. Your example of brake lights is good, as it provides a cue as to what is likely to happen.
It also means that the high latency of a visual processing system is less of a problem, as the model can be updated by the lidar. Camera picks up a car, the model attaches it to the pointcloud it thinks is the car from the lidar/radar. When the lidar/radar updates it can create a prediction of where the visual system should look.
> detecting perpendicular cars vs parked cars at that range?
You don't have to. lidar gives you a point cloud, those points can be roughly translated into hard surfaces. if a hard surface is in your path or predicted path, take action until it isn't. Dealing with pointclouds for object avoidance is much much more computationally simpler, than having to infer 3d from either structure from motion, stereo or both.
https://github.com/raulmur/ORB_SLAM2 if you look at this slam system, the pointcloud it generates is very sparse compared to a lidar: https://youtu.be/W3ELziPYn5k?t=13
But as I said before, you need other sensors to get other data/corroborate world model.
>lidar was quite sensitive to dust, rain, snow, and fog.
It is indeed, like any other sensor
> Will weather cause the lidar to decay in similar ways to human eyes?
Most lidars operate in the far infrared. so will handle decay differently. Depending on frequency it'll either be sensitive to moisture, or not at all.
I see this repeated over and over in this thread, elsewhere on HN, and in many other places on the web.
But no one explains what they mean by a safe driver.
There are other factors at work than mere human-ness. For instance the US has more than double the number of fatalities per unit vehicle distance than the UK, more than triple per head of population. In both countries the driver is a human yet the disparity in outcome is huge.
Perhaps whatever it is that causes this sort of difference should be addressed rather than insisting on a new an inherently unproven technology.
Or, at least, give us some numbers so that we can tell what you mean by safe.
See https://en.wikipedia.org/wiki/List_of_countries_by_traffic-r...
You have just given numbers: when autonomous cars beat those numbers they are better. Until then we should be careful.
Why not? We've been seeing TV footage of cameras seeing perfectly well in the dark for decades. Why is there a problems with using them for automatic driving at night?