Hacker News Reader

Lidar doesn't use visible spectrum light. They're usually infrared, so Musk's quote makes no sense.

You're making a lot of strong assumptions to draw your conclusions: that commodity cameras and radar can compete on measurement accuracy with lidar systems, and that lidar costs won't decrease with additional investment (we've already seen costs decrease, by like a factor of 10x in less than a decade). The power questions also aren't cut and dry: if you need extra in-vehicle GPUs to support the radar+camera approach, you may well be using more power than a lidar based approach.

There's also no real requirement that AVs operate in snow or dense fog. Those are only considerations in certain climates in certain seasons. You don't actually need the safety system to pass the safety tests (that don't currently exist) in worst case conditions if the vehicle works anyway. Why optimize for the worst case first?

I'll respond clearly: We're multiple computer vision leaps forward away from what Elon needs for success. They're easily half a decade behind Lidar based systems. And people die as a consequence of putting those systems on the road.

> The power questions also aren't cut and dry: if you need extra in-vehicle GPUs to support the radar+camera approach, you may well be using more power than a lidar based approach.

All current lidar-based approaches I'm aware of also supplement with radar+cameras. LIDAR isn't sufficient in isolation. GPUs consume way too much power, there's no way you can cavalierly just add more of them as a scaling solution.

> You don't actually need the safety system to pass the safety tests (that don't currently exist) in worst case conditions if the vehicle works anyway. Why optimize for the worst case first?

Not even sure how to interpret such a statement.

> I'll respond clearly: We're multiple computer vision leaps forward away from what Elon needs for success. They're easily half a decade behind Lidar based systems. And people die as a consequence of putting those systems on the road.

Custom ASICs for ML that Tesla is building is fairly well-understood tech at this point. High-end smartphones have used similar tech for years now (though perhaps an apples-oranges comparison).

Only criticism I would level against Tesla's current approach is their overly optimistic time estimates, and hand-waviness about the complexity of solving certain very complicated edge-cases. However their technological approach is quite sound.

> All current lidar-based approaches I'm aware of also supplement with radar+cameras. LIDAR isn't sufficient in isolation. GPUs consume way too much power, there's no way you can cavalierly just add more of them as a scaling solution.

Yes, but they don't need to get point-cloud level spacial data from a suite of cameras. That takes more compute power to do at 60fps accurately with cameras than with a lidar.

> Not even sure how to interpret such a statement.

Let me rephrase: You don't need the vehicles to work in worst case conditions at all, if economically they're still a success if you don't allow them to run in those conditions.

If Waymo or Cruise who whomever has self driving taxis in 2020 deployed in temperate cities, and Tesla doesn't have L4 autonomy until 2024, at which point it also works in a blizzard, it doesn't matter if they're cheaper and more effective them. Tesla will have already lost.

> Custom ASICs for ML that Tesla is building is fairly well-understood tech at this point.

This is one small piece of the puzzle. You also need algorithms that Tesla doesn't appear to have, and (camera) hardware that Tesla claims to have but others seem to agree can't support what they want.

> If Waymo or Cruise who whomever has self driving taxis in 2020 deployed in temperate cities, and Tesla doesn't have L4 autonomy until 2024, at which point it also works in a blizzard, it doesn't matter if they're cheaper and more effective them. Tesla will have already lost.

If those self-driving taxis are not available for retail purchase and cost their operators over 6-figure sums to add to their fleet - and Tesla have it working in their promised $35k Model 3, then Tesla will win in the long-term.

Also, remember the car industry moves very, very slowly (no pun intended). The Model 3 has been out for over a year now and has barely captured a fraction of its possible market.

Consider that people will buy/lease a car for individual use for 3-5 years before replacing it: someone buys a normal car in 2021 because they need one at the time, but will still buy Tesla’s FSD car when it comes out.

The “autonomous taxis will replace individual car ownership” trope only applies to hyper-urbanised environments where parking-spaces are luxury lifestyle accessories - and where people are already well-services by public-transit infrastructure.

Finally: people who try an autonomous (but unaffordable for exclusive individual use) taxi service in 2020 might be so impressed with the experience that they vow to buy the first available individually purchasable autonomous car - even if that happens until Tesla’s 2024 models come out - Tesla still wins.

By analogy, consider that Tesla is probably the /hottest/ car brand in the world today - and they did it without any traditional advertising - and they got started over 10 years ago. Enough people influenced by reviews and YouTube videos of the Roadster and 2012 Model S translated that into real money being spent on Model 3 buys today. That’s an anticipation gap of at least to 7 years - that’s impressive. Can you imagine people waiting 7 years for an “affordable” version of a luxury product? Why weren’t any of the other manufacturers doing anything to meet this clear demand for physically attractive EVs with over-hyped autonomous capabilities?

(Disclaimer: I own a Tesla Model X)

> only applies to hyper-urbanised environments

That's where most of the population lives in western countries (both US and EU have ~10% of the population in the largest 10 cities and that's without the well connected metropolitan areas which probably at least double the numbers). So if the futuristic predictions of shared vehicles actually materialize the market for the "personal" vehicle would shrink and manufacturers will have less incentive for whatever they produce for the "personal car" slice of the market. Unless we also get a decentralization trend and people start leaving urban centers.

> Why weren’t any of the other manufacturers doing anything to meet this clear demand for physically attractive EVs with over-hyped autonomous capabilities?

While I agree with the "physically attractive EVs" (although for me it's more of a nice to have than a must), the second part is pretty cynical. You're asking why aren't manufacturers knowingly killing people to sell more cars. Well they are. Some promise you clean diesel, some promise you self driving. Both missed the mark a bit. And both of those promises sold lots of cars.

>> I'll respond clearly: We're multiple computer vision leaps forward away from what Elon needs for success. They're easily half a decade behind Lidar based systems. And people die as a consequence of putting those systems on the road.

>Custom ASICs for ML that Tesla is building is fairly well-understood tech at this point. High-end smartphones have used similar tech for years now

joshuamorton is right and you didn't really address the point made. It isn't about understanding the hardware, it's about understanding the software. You point me to any individual who can tell you concretely how a specific convolutional neural network (CNN) arrives at a given solution and I'll take this all back.

We know so little about how CNNs work, and plane old neural networks (NNs) for that matter, it's embarrassing. They're prone to adversarial attacks, not only that, you can successfully mount the same attack against _any_ NN that was trained on a common dataset. We don't know how to effectively defend against these yet in a white box setting.

We have no idea what the solution space looks like. We barely understand why one of the most simple optimization algorithms outperforms almost all others on at these tasks. We barely understand why randomizing data visitation results in solutions that perform completely differently.

The counter argument goes something like "we tested it on a big eval set and it aced it" Well I'm here to tell you that things change. Assumptions made in creating that eval set might not generalize to all real-world cases. And in this case bad assumptions result in death.

As someone who has spent a good portion of my life working on this stuff I've learned that sometimes (most of the time) the best choice is the obvious, known, simple one. The fact that a good portion of cars may be controlled via something we know so little about should worry you.

On a different note, I think you're also missing why Tesla have opted to invest in a percept suite that doesn't use LIDAR. That reason is cost. Tesla needs to sell cars now. They and their customers cannot afford to put LIDAR on their current platform. At the same time they need to move the metal and they know their customers want AEVs. I think their strategy is sound from a purely business cost/benefit analysis. It's a risk but from a financial perspective a good one, because if it works they hit pay dirt.

> The fact that a good portion of cars may be controlled via something we know so little about should worry you.

I bet that we know much, much more about CNNs than about our brains. And today pretty much all the cars are controlled by something that we don’t know basically nothing about. Why this doesn’t worry you?

The fields of neuroscience and psychology are far more advanced compared with the infantile field of deep learning.

Humans also learn independently from one another which means outlier events aren’t as much of a problem. You have the ability to observe the world around you, say you see an accident due to some rare weather event, and learn some abstract lesson extremely quickly. A CNN has to learn this over millions of samples out of band.

EDIT: clarification in last sentence.

Based on the number of actual reproducible controlled experiments done on CNNs, I'm not convinced the field is less advanced than psychology at this point.

Well, we know quite a bit about how humans perform at the relevant levels of abstraction, and that's the level at which driving automation systems should be evaluated. They are not there yet, but there's no reason to believe the problems are insurmountable.

> I think you're also missing why Tesla have opted to invest in a percept suite that doesn't use LIDAR. That reason is cost.

Even beyond pure cost, one problem Tesla has is that they already SOLD tens of thousands of FSD option packages, predicated on NOT needing to retrofit the cars with LIDAR.

So arguably, Tesla might be worse off if they deliver a LIDAR based FSD (and need to retrofit tens of thousands of cars, or pay off the owners), than if they just plod on with a camera based FSD that never quite works safely.

>one problem Tesla has is that they already SOLD tens of thousands of FSD option packages

It's always seemed sketchy to me that they're selling FSD option packages when they don't know when FSD will actually happen or whether the current in-car hardware will be adequate to support it once it does happen.

He's likely banking on a bimodal distribution where FSD either happens very soon (within a year or so) in which case the hardware likely will be fine OR not for 5+ years in which case these cars likely will not be on the road any more and it becomes a moot point.

This is what I meant, I should have been more clear.

> You don't actually need the safety system to pass the safety tests (that don't currently exist) in worst case conditions

I do not agree, making the systems work in all conditions is the hard part of true automated driving.

Also, worst conditions happen 1/50th of the year ( guess). So it's not that rare

Safety systems certainly need to work in all conditions, but “work” in this case may mean refusing to activate in conditions outside its design range. It’s fine for a self-driving system to refuse to drive in a blizzard; it’s not ok for it to try and then fail to drive in the blizzard.

Refusing to drive could work when starting out, but handling changing conditions on the road is harder.

A scaled request for the driver to take over as conditions get worse can train the driver not to use the self-driving system in adverse conditions, so hopefully it wouldn't have to refuse (unless the driver is negligent).

Getting back to the OP, Musk may have a point: people are terrible at evaluating risk for low-probability/high-consequence events like a car accident, so LIDAR might lose in the market even if it is worth it. But if there were standards for when the car asked the driver to take over and LIDAR is able to pester its drivers less often because it is more capable, then perhaps LIDAR can justify its place in the market.

Moderate rain is not an acceptable condition for cars to refuse to drive. Especially in the middle of long trips.

This means self driving cars must also operate without LIDAR, though the safety advantage when it is operating can still be a net gain.

> Moderate rain is not an acceptable condition for cars to refuse to drive. Especially in the middle of long trips.

It is absolutely acceptable for a self-driving system to refuse to drive in any conditions it can’t handle, but it must also deactivate in a safe way when such conditions arise during operation (e.g. pull over to the side of the road if the driver hasn’t positively acknowledged resuming control).

Commercial viability of any given system is a separate issue, but it’s pretty well accepted moral responsibility to not accept control of a vehicle if you’re unable to operate it safely, regardless of the consequences of that refusal(1). I see no reason to not hold consumer-facing self driving systems to the same standard. Otherwise, they require some specialized training for the operator to be able to recognize the situations in which they are safe to use.

(1) Actual life-and-death situations change this a little, but future availability of rescue personnel and equipment generally weight the conclusion towards operating safely in those situations as well.

The pulling over to the side of the road solution doesn’t work at scale. What happens when it starts snowing on a 10 lane freeway during rush hour and 50% of the cars are self driving with this limitation?

It’s unlikely that, in a region that gets such snowstorms, a self-driving system that can’t handle them reaches 50% market penetration. That seems more like a “we’ll cross that bridge when we come to it” scenario.

Rain and fog are much bigger issues for these systems than snow. Being unable to sell cars in places with rain does not leave many options.

However, for rare events it’s very possible for individuals to be fine with something and only collectively do you end up with major problems.

OK then QED. The point some are making is that with at least the current type of lidar, that bridge may not exist. It may make more sense to devote resources to better radars and radar processing.

And I’m not disputing that at all. None of my statements say anything about any particular self-driving technology as I’m not an expert in the various technologies. My point is that what’s “acceptable” is a fundamentally a moral stance, and stated the minimum bar I believe all drivers (automated or not) need to meet.

This is simply a constraint that any system needs to work within, and it’s entirely possible that precludes the commercial viability of LIDAR-based systems. It’s also possible that there’s some niche market that fair-weather-only systems can be successful in long before the general problem is solved, and we shouldn’t artificially throw those out as “unacceptable” when there’s a reasonable framework for them to operate under.

Unless you're driving on a road, and heavy snow develops earlier or more severe than you and the weather service anticipated. Instead of going the next 2-3 miles safely to your exit, your car will decide it's best for you to strand you in the middle of the motorway in white out conditions until the weather passes? Color me skeptical.

> Instead of going the next 2-3 miles safely to your exit

You’re making an assumption here that the system is capable of continuing to travel safely. Obviously being safe at home is better than being stopped on the side of the road, but that’s not the choice you’re actually faced with. Similarly, a system that can operate safely in adverse conditions is obviously better than one that can’t.

I know in my good-old human-driven mode of transport what choice I would make.

I don't give automated driving any allowances, if it can't do what I do, it doesn't belong on the road.

> I don't give automated driving any allowances, if it can't do what I do, it doesn't belong on the road

In that case, you’re almost certainly grouping a lot of current and safe drivers in the “it doesn’t belong on the road” category. Not having lived in a place where I’ve needed to deal with white-out conditions myself, I doubt I’d feel comfortable continuing to drive. I know this about myself, though, and am likely more conservative than you about canceling or rescheduling a trip when such conditions are a possibility. This is the same bar that I’m proposing automatic driving systems need to meet; if that means they only work on cloudless days with 0% chance of rain, so be it. In practice, this means there’ll need to be some way for a human to take control for quite a while yet.

One industry that is very close to full autonomous operations is aerospace. In a controlled environment with less things to run into. And even they disengage the Autopilot in certain conditions. Why would we aim at full autonomy for cars in an uncontrollable chaotic environment with a lot of stuff to hit under all circumstances? In some countries you are not allowed to drive with summer tires in winter for safety reasons. So why not limit self driving capabilities in similar fashion?

From the business perspective, this would not be desirable because it destroys a lot of prospective business cases for self-driving vehicles. Basically all of them that are as-a-service. What good is a taxi service that stops working during bad weather? That's even when there would be more demand for it because people don't want to walk or bike anymore.

Says a lot about some of these business ideas, doesn't it?

But in all seriousness, if self driving cars are not ready as fast or as performant as expected a lot of these long term bets may be in a lot of trouble. This potentially includes Uber, Lyft, Tesla by self-declaration, and others.

And safety isn’t limited to working in harsh conditions. Mistaking a picture on a billboard for a real object, or mistaking a truck color for the sky can also get you a crash by clear weather.

The crutches analogy isn’t a very good one. If you ask a doctor, he will probably advise you to use crutches until you can stand firmly on your own two feet!

Those don't sound like things Lidar would have a problem with. The color doesn't matter only whether the surface is reflective (basically anything not matte black). But otherwise objects are easy to detect and distance.

The problem with weather is different as the individual droplets bounces the Lidar light before it hits the object.

My point is rather that those are things CV would have problems with that Lidar disambiguate.

And yes there are conditions where neither work well, and frankly where humans barely function either. Much of driving in heavy fog / heavy snowing is really a leap of faith at low speed.

>They're usually infrared, so Musk's quote makes no sense.

Many of Musk's statements don't hold up to the scrutiny of professionals or specialists.

> So in the end I think Elon will be proved right. Those currently investing in LIDAR-based systems will eventually ditch it for purely practical economic reasons. Those that don't will be completely destroyed in the open market.

High quality cameras are insanely complex pieces of electronics and optics. Ditto for processors capable of doing quality image recognition. Large scale manufacturing has made them cheap nonetheless. LIDAR is relatively niche, but if it proves useful to deploy it at scale, I'd expect costs to drop very significantly. The underlying technology uses very simple physics (relative to the algorithmic complexity of image recognition); seems like a solid basis to build a sensor off of.

> LIDAR is a crutch for bad software

You could invert this and say that high precision image recognition is a crutch for ill-suited hardware. The final combination is a product of hardware and software. If LIDAR is currently too expensive or energy-intensive to compete cost-wise at acceptable safety levels, that's one argument, but saying LIDAR is a crutch is just moving the goalposts from "good system" to "cheap hardware".

[edit] Also, just want to point out that RADAR resolutions are way too low to operate a vehicle safely (never mind road signage or other things).

> The underlying technology uses very simple physics (relative to the algorithmic complexity of image recognition); seems like a solid basis to build a sensor off of.

The crux of the argument is you will still need the algorithmic complexity in the end with or without LIDAR, so it doesn't add any advantage.

I'll just paste the quote FTA:

> "Lidar is really a shortcut," added Tesla AI guru Andrej Karpathy. "It sidesteps the fundamental problems of visual recognition that is necessary for autonomy. It gives a false sense of progress, and is ultimately a crutch."

> The crux of the argument is you will still need the algorithmic complexity in the end with or without LIDAR, so it doesn't add any advantage.

The approach described still uses the same ML on a Lidar data representation, but with an extra step on image recognition to put it into that data format. Image recognition only adds computational complexity post-hardware-sensor because you first need to generate a Lidar-like 3D map. So that is conceivably an advantage for Lidar in the long run given that its method for making the 3D representation of the space (upon which the ML runs) is dirt-simple physics-wise—much simpler than the image recognition algorithms and camera electronics used by image recognition. It's just not cheap yet because the market for processors and cameras is huge and the products have become incredibly sophisticated and cheap. Again, hard to imagine an idea as simple as Lidar not getting much cheaper with scale.

> I'll just paste the quote FTA

My point wasn't an ad hominem one, so I don't think the fact that an expert like Karpathy said "crutch" changes the fact that it's slanted phrasing. But given that Karpathy works for Tesla and has a vested interest in assuaging investors and consumers, it makes sense why he used slanted phrasing. "Lidar is a crutch" makes Tesla sound more visionary than "we don't want to pay for Lidar because it's still too expensive and we think we'll be able to rival Lidar with image recognition." It's a nice way to subtly jab at competitors who are investing in Lidar and frame it as if Lidar was the tech that had catching up to do (when, in fact, the opposite is true).

Since the key point of the article is that ML algorithms work better with Lidar data representations, it's pretty hard to see it any other way. They both go to the same intermediate data representation, and Lidar still wins in a head-to-head. Again, you can argue validly that cameras + image recognition will be good enough, but calling Lidar a "crutch" seems like pro-Tesla spin by a high-up Tesla employee whose job is to make Tesla look good.

While your comments have gone a long way towards changing my mind, Karpathy's comments are a bit rich, given that Tesla's systems could have benefited from a crutch to help avoid running full-tilt into large obstacles.

If all you use is words, it is of course easy to omit that the spatial resolution of radar is barely enough to tell there are one or or two vehicle sized objects in front of you, maybe one to the side, and they better be moving.

To compare it to LIDAR is ludicrous.

Ford did some research to improve Lidar for use in the rain/snow using a filtering algorithm.

> Ford’s autonomous cars rely on LiDAR sensors that emit short bursts of lasers as they drive along. The car pieces together these laser bursts to create a high-resolution 3D map of the environment. The new algorithm allows the car to analyze those laser bursts and their subsequent echoes to figure out whether they’re hitting raindrops or snowflakes.

https://qz.com/637509/driverless-cars-have-a-new-way-to-navi...

Still seems less than ideal but I'm curious if that will ever reach somewhere useful.

FWIW, lidar is very effective in finding the surface below forest canopies. In fact, it will often identify the underbrush, in addition to the canopy top and the surface.

Aren't there still problems with multiple active sensors sweeping the environment?

I remember it being a problem in cars that used Lidar but cannot find the info anymore.

I think Lidar could still be of help and even the perfect software can use any form of sensory redundancy. But I agree that there might be alternatives.

edit: A laser is probably a lot cheaper than camera and imaging dsps if comparable production scales are reached.

>So in the end I think Elon will be proved right. Those currently investing in LIDAR-based systems will eventually ditch it for purely practical economic reasons. Those that don't will be completely destroyed in the open market.

Maybe, but there's no reason to leave a local maxima until you actually have something better.