Hacker News Reader

Comment by barnacled | Hacker News Reader

I like Filip Piekniewski's take on this [0] - deep learning is appropriate for environments where the cost of mistakes is low (or the errors are reversible) and you're able to 'average them out'. Self-driving cars are absolutely not in that category.

"If your errors are irreversible and you can't average them out, and your application is in open domain where there will never be a shortage of out of distribution samples, you are royally screwed. That is why I'm so skeptical of self driving cars, since that is exactly the application described."

For me it's obvious the real issue are the edge cases - of course a self-driving car does fine in 99%+ of cases, but those small % of outlier cases are the ones where people potentially die. I suspect an entirely different approach is required to supplement/supplant the deep learning one and fully autonomous vehicles are still quite some way off.

I do like all of the driver assist advantages coming out of this work however! I don't think you can underestimate the value of having technology that works _with_ a driver until the point where self-driving cars become viable.

[0]:https://blog.piekniewski.info/2020/06/08/ai-the-no-bullshit-...

View on HN

I can see it being possible to "average out" the mortality of self-driving cars when you consider how high road mortality is. But they won't be the same deaths, some would be things a human driver would avoid. We generally define the safety requirements of self-driving cars as being close to airliner reliability but also generally accept that roads, cars and drivers have much lower standards than that right now. So in this environment we can very soon be stuck in an uncanny valley where self-driving cars would already be a net positive in reducing road fatalities but at a level that is still generally considered unacceptable.

I think what further compounds this is that normally responsibility for accidents is 'distributed' across the population. whereas with autonomous driving it's going to be a single/a few companies shouldering it all. Being even responsible even just for 10% of total road fatalities (~4000 deaths per year) would be absolutely ruinous if they're held liable for them.

There's (arguably) the limited exception of pharmaceuticals--and, even there, there's a government fund to handle people who get sick or die because of vaccines. But it's hard to otherwise identify products used by consumers that, when maintained and used properly, sometimes randomly kill people and we shrug our shoulders and say "Stuff happens. Still a net benefit to society." We certainly don't do it today in the case of manufacturing or design defects in vehicles.

Responsibility for accidents is concentrated in insurance companies. Nothing substantial will change.

I think firstly it's going to be tough with long-tail edge case deaths to really be sure the numbers will be less than human drivers until self-driving cars are unleashed in sufficient numbers.

Secondly, even if you're right (and you may well be) - imagine that a common edge case was a small child jumping out in front of the car. Would we want to accept a vehicle that consistently ran them over? The raw numbers might be better but that doesn't fully paint a picture of what those numbers would look like.

I also think the 'will randomly kill people, but rarely' aspect of self-driving cars will be majorly off-putting on a very human level.

And finally there's a perception issue here - the self-driving hype is around the idea that the car is effectively being driven by a far better driver than a human, the second it turns out it does really crazy things sometimes (doesn't even have to involve fatalities necessarily) is also going to put people off I think.

I personally prefer the idea of driver assists (responsibly developed so drivers aren't just distracted) aiding driver safety while we work on a supplemental self-driving solution, or perhaps as discussed in the article create some infrastructure along roads to make things more certain.

I am far from being against self-driving cars, I just feel the timeline is longer than currently hyped.

I agree completely. My take from the situation is much more that we are far far too lenient with our standards for cars, roads, and particularly drivers. We've set the bar so low that the technology has a good chance of clearing it soon but only serve to reveal how actually uncomfortable we are with such a low bar that we've normalized over time.

In many parts of North America you cannot function in society without a license. Therefore, we have sympathy for people because are we going to ask people who aren't good drivers to put their lives on hold on the off chance they kill someone? Better to let them all drive and vilify the unfortunate ones that do. However, with machines they have no personal lives so they are afforded no slack. Humans get to make mistakes because they are only human, but machines get no such affordances... even if the net benefit is there.

Another point might be attribution. It's horrible when a driver is tired, drunk or simply misses something - but on a human level, you can understand what's going on. Also, you know the guy at fault and that he will be punished (at least, somewhat).

An autonomous vehicle, in comparison, is not relatable at all. Some equation delivered some wrong result, maybe it was something in the training set, so that's why the car suddenly drove into you. Bad luck. If you're lucky, you get some damage payments from the car company, if you happen to have enough money for a very good lawyer.

Sure, as a technical person, I understand some of the possible errors, but it still feels like a complex system that one cannot relate too with hardly understandable edge case behaviour. That's probably why we're so much more lenient with humans.

Might be interesting to integrate an AI based self.drivinf system with several additional driver assistance systems, each using fundamentally different algorithmic solutions, and perhaps even some different sensors.

>when you consider how high road mortality is

Except, depending upon your perspective, it really isn't most places. In the US it's about one fatality per 100 million miles driven and that includes driving under difficult conditions like snowstorms. People drive a lot so that still adds up to a lot of deaths per year. But the rate is not really all that high.

That rate is on the order of 1000x worse than airliners which was the comparison as that's the level of reliability people are expecting from self-driving cars. A self-driving car that's 100x worse than the airliner standard would still save 90% of all road fatalities. And yet saving 30k lives might still not be enough for people to accept such a lower standard compared to an airliner.

https://en.wikipedia.org/wiki/Transportation_safety_in_the_U...

That seems an unrealistic bar especially as a hypothetical self-driving vehicle will be co-existing with human-driven cars even on main roads for the foreseeable future. Furthermore, airliners are flown by a pair of trained professionals, within an air traffic control system that minimizes their exposure to most serious threats, and are mostly (some bird strikes, sensor failures, etc. notwithstanding) flying under conditions that are fairly predictable. OTOH, I doubt a self-driving vehicle will be all that better at dealing with a deer jumping out in front of a car, unexpected road debris, or black ice all that much better than an alert human driver. (And possibly worse given that a human driver may pick up on certain cues that an automated system might not.)

I agree with most of this, except for the unexpected rapid-response examples you list - I would expect an autonomous collision-avoidance system to react faster than a human being when faced with a deer in the road or someone running a light (otherwise - it's not a very good collision avoidance system!)

It can react faster to an object. But, under at least some circumstances, a human might recognize potentially questionable road conditions in a way that an autonomous vehicle would not. (Of course, presumably the human could also tell the autonomous vehicle to slow down.) And I've definitely seen animals at the side of the road looking they may be inclined to run out.

Overall, I don't really disagree on net though.

Something here is unrealistic, that's the point. I'm not so sure it's the reliability threshold for self-driving cars though. I'm more amazed at our continuing acceptance of the risks in driving. Having two pilots, air traffic control, maintenance standards, etc, are all regulation driven. Why do we have all those regulations if we're willing to accept 1000x worse in cars with much more lax regulation?

> OTOH, I doubt a self-driving vehicle will be all that better at dealing with a deer jumping out in front of a car, unexpected road debris, or black ice all that much better than an alert human driver. (And possibly worse given that a human driver may pick up on certain cues that an automated system might not.)

I'd expect the opposite actually. Humans are incredibly slow reacting in the best of circumstances and they often drive distracted. They also suck at reacting even when they do which is why we now have automatic systems to reinforce braking in presumed emergencies as humans don't consistently apply full brakes in emergency situations. Collision avoidance in a few milliseconds with reliable reactions should make quite a few of those scenarios much better.

>Why do we have all those regulations if we're willing to accept 1000x worse in cars with much more lax regulation?

I think it boils down to "because we can and still have a viable commercial airline industry." (And general aviation is much riskier than commercial airlines.) Whereas were we to vastly increase the training, maintenance schedules, monitoring of driver alertness, and requirements to have all the latest safety features, you're probably effectively eliminating driving for most people who can't just hire a professional driver/car.

For many things, we accept that we could make things safer if we wanted to. But were we to do so, we'd effectively be banning the activity or at least severely restricting it. (Which is probably one reason why we don't really have flying cars.)

> as that's the level of reliability people are expecting from self-driving cars

Why do you think this? It makes sense to me intuitively that people would be willing to accept more death causes by other people than more opaque and unfamiliar autonomous systems, but I haven't seen any hard data on this and am curious. Is this mentioned somewhere in the Wikipedia article that you linked or did you just share that to demonstrate the other number you mention?

The Wikipedia article is just for the fatality statistics. The expectation of airliner reliability for self-driving cars is what I get from the discussions around this. Here on HN and in other places. Maybe it will be lower and the uncanny valley will be smaller but I wouldn't bet on it. People are much more afraid of flying than of driving even though driving is much riskier. And that's because you're much more willing to accept risks when you think you're in control of them. Self-driving cars will take that illusion of personal control away.

Gotcha, OK. Yeah, I agree with you directionally.

This statistic is in terms of passenger miles, but airplanes travel much faster per car, and airplane journeys are normally much longer than a typical car ride.

So, planes might be around 500x better per pax-mile, but per pax-journey they're just about 5x better.

Incidentally, the fatality rate for self driving cars should be also around 5x better (per vehicle mile) than human drivers for wide spread acceptance, according to an 2018 study.

https://www.sciencedaily.com/releases/2018/05/180530132959.h...

Per-distance statistics may stretch the risk perception as people use planes for different trips than cars. But per journey statistics seems a much bigger error in the other direction. Fatalities with airliners are pretty close to 0 per year whereas 30k+ people a year die in car accidents. Most people know of someone that died that way whereas any airplane accident is national news. 5x really understates that difference. I don't think per-journey numbers are reasonable as a comparison.

The study showing 5x is quite neat, thanks. I expect this to be a floor for the difference though. Once some of the deaths with self-driving cars are shown to be things that human drivers would easily avoid the requirements will be higher, even if in total 5x better has been achieved. For widespread acceptance I'd bet on 10x as a minimum and 100x at the extreme, depending on what types of accidents are in there that shape the perception.

I think the best way to reduce those outlier cases in this phase is to just focus on self driving on the freeway. That's where it has the most value for me, taking over for the long boring hauls.

On the freeway you minimize a lot of those wildcards like bikes and pedestrians, everyone is going about the same speed, and there are only a few places you can enter and exit.

We don't need to jump all the way to self driving in SF, and have people point out all the things that can go wrong, since theres a huge gain just from being able to do it on the freeway.

I totally agree with you.

So why hasn't this happened yet?

I believe this is because what we have here is not a technical problem, but primarily a regulatory and legal problem. If the political leadership of the NHTSA had the will to make highway self-driving happen, they could publish a draft safety and liability standard.

The technical problem here is that while shuttle and robotaxi systems can go slow enough as to make fatal accidents rare, as you said, the most consumer benefit potential is for a highway self-driving system that would not require active monitoring like Autopilot does.

The political/legal question is: when fatal accidents occur, who has the legal liability? If it's the manufacturer, then why would they accept the risk? If it's the driver, then who would actually go to sleep and let their robot potentially kill people? Especially given the problems that Tesla autopilot has with stationary targets in the road.

It's possible that the key innovation required here is not anything technical, but an insurance product that would allow the manufacturer to assume the liability, but be insured for damages. This insurance product would involve a deep technical evaluation of the self-driving system to get actuarial-level probabilities of mortality.

On the NHTSA website, there are many >50 self-driving test sites, but only 3 which are classified as "highway". The most common vehicle type is "shuttle". A full self-driving airport shuttle, with purpose-built road markings, makes a lot of sense as a goal achievable with today's technology, and is far more likely to kill people due to its controlled built environment and slow speeds. https://www.nhtsa.gov/automated-vehicles-safety/av-test-init...

My proposal is that we treat liability in the self driving car space similar to the way we have a specialized liability framework for vaccines [1]. Similar to vaccines, self driving cars would lead to reduced deaths on aggregate, but there are cases where the technology fails and people are injured/killed. In the case of vaccines, consumers cannot sue the manufacturers using regular courts but instead there is a separate system where the standard for liability is much higher.

[1] https://www.linkedin.com/pulse/driverless-car-stopped-lawyer...

   > In the case of vaccines, consumers
   > cannot sue the manufacturers using
   > regular courts but instead there is
   > a separate system where the standard
   > for liability is much higher.

So when your car runs me over, I cannot even get my medical bills paid?!

And you wonder why there is no political will behind this?

> For me it's obvious the real issue are the edge cases - of course a self-driving car does fine in 99%+ of cases, but those small % of outlier cases are the ones where people potentially die. I suspect an entirely different approach is required to supplement/supplant the deep learning one and fully autonomous vehicles are still quite some way off.

Why is that every time the safety of self driving cars is brought up, people appear to ignore that human drivers are not 100% perfect. There are on average of 104 deaths on the road each day in the US. if we can get that number to 50 people, that's a net positive.

Self driving cars don't need to be perfect, just better than us.

So the question shouldn't be "are SDC perfect yet?" but "are they better than us?"

Is the manufacturer gonna take the liability for those deaths?

It seems obvious they'll have to provided the vehicles are being used and maintained in accordance with manufacturer instructions.

Would you get in a vehicle if you could be charged with manslaughter for killing someone through absolutely no fault of your own?

Conversely, can imagine society collectively throwing up their hands in the event of a death caused by an autonomous vehicles and going "Them's the breaks. It's all for the good of society."

If self-driving cars are marginally better at driving than humans, but people take more rides in them out of convenience, it's easy for the technology to result in a net increase of human deaths.

I think you misinterpret what I said, it is the edge case scenarios where fatalities tend to occur. Of course in humans that can be due to drunkeness or distraction or confusion, nobody is claiming human drivers are perfect, but if it is the 'weird' situations that tend to result in accidents and deep learning tends to make mistakes in 'weird' circumstances, that doesn't fill me with confidence.

Also it is not at all obvious that self driving cars will do better than humans. Perhaps they will, but it isn't just about numbers. As I alluded to in another comment, imagine if they had 50% of the human death rate but it was all small children and occurred seemingly randomly. Do you think that would be acceptable and a better outcome? These things are less black and white than they might seem.

Another factor is that with deep learning it is fully a black box. It is very difficult to understand WHY a particular course of action was taken, whereas in human cases you can usually determine this. The why might also be limited, e.g. 'human child was mislabelled as stop sign' or similar, which just causes further questions.

The fact that you have scenarios the car will definitely not know how to handle combined with a deep level of inscrutability makes it difficult to have confidence in issues being resolved or the technology improving on a safety basis.

I don't think if all cars were replaced by self driving cars fatalities would go down. In any case there is little evidence to support that. So I am skeptical of this claim that people are being too hard on the self driving cars.

SDCs will never be 100% safe because at the end of the day, the underlying task is still to push around a couple tons of metal on heterogeneous roads at speeds requiring dozens of feet of stopping distance.

But if it can get to half the fatality rate of human drivers, that'd be a huge win. And unlike human drivers, every fatality involving an SDC can become a fleet-wide software improvement to, ideally, prevent that failure from ever happening again. That's not something we can do with people; nobody but traffic engineers and wonks read NHTSA reports to find out how accidents could have been avoided.

I suspect the outliers that confound machines will also frequently confound human drivers, so a lot of that is a wash.

The only way to remotely look at this meaningfully is in aggregate. Deaths and losses per million trip-miles.

This is where the momentum will come from. There will be a day where machines reach and surpass the human level, and we'll see a huge momentum towards penalty-boxing human drivers. I fully expect to call Progressive in 2040 and the hologram Flo will tell me that it costs four times as much to insure me if I have the audacity to retain a steering wheel.

I also think there's a snowball factor. Self-driving cars are going to be way more consistent and predictable than human drivers. That only amplifies as more and more vehicles on the road are self-driving. Once you get to 90% self-driving, I could imagine things like increasing speed limits and dynamically organizing convoys for efficiency.

Right. We’d probably be better as a society if we were interested and willing to do some infrastructure projects to alleviate the burden of driving cars. I really wish Elon’s boring company were further along and getting much more headwind for this reason.

I am extremely skeptical of the boring project. The claimed money saving (a 10x improvement vs. every single tunnel digger in the world without any evidence to back it up?) the safety issues of a car breaking down in a tunnel or being brought up and down, the enormous cost with digging deeper tunnels, etc. etc.

Road tunnels exist and have been built all around the world, these kinds of ideas have been tried and ultimately general underground road tunnels are just not economical or particularly safe.

Thunderf00t has done a series of videos debunking these claims [0,1,2]. Unfortunately Elon has a habit of frankly lying which I think does serious harm to public perception of these things.

These kind of infrastructure problems are just hard and have a ton of trade offs, there is no silver bullet.

[0]:https://www.youtube.com/watch?v=HBtL3qDvdZc [1]:https://www.youtube.com/watch?v=0ezF7NmwQZs [2]:https://www.youtube.com/watch?v=Hyy-19qyNWE

Why tunnels instead of the Japanese/German solution of ramps/viaducts?

In the same vein, why underground instead of light rail?

See, open air systems like those are much easier to service. Removing a crashed car from that requires just a big crane.

Underground doesn’t get affected by weather. I wonder whether simply having more tunnels side by side would help for servicing issues. Small, numerous tunnels instead of large tunnels.

Building wide tunnels is very hard and nullifies the main advantage of having the tunnel in the first place, which is easier routing.

Cost also vastly increases, likelihood of total failure as well. Big tunnels need big supports and big service ducts.

Because tunnels can go under buildings, roads and other tunnels with very little interference.

Bridges can also go above bridges, probably cheaper and safer as well

Rocket launch technology was at a standstill for 30-40 years until Elon Musk founded SpaceX. 15 years later they built reusable rockets. The cost of putting something into orbit is likely to drop by a factor of 100-1000 when SpaceX's next rocket is up and running sometime in the next few years.

It seems like such a waste of time for people to try and debunk these claims from people who are very competent and have access to the best in people and technology. Do they want a world where people don't try new things and make great improvements? One should really encourage people with a track record of getting people together to do amazing things to continue to do so. Elon Musk has stated that he thought that SpaceX had a 10% chance of success. I'm glad he could look at that chance of failure and tried anyway.

I don't follow rocket technology closely, but is SpaceX really unique in regard to reusable rockets? Didn't Blue Origin do each milestone (announcement, vertical landing, relaunch) before SpaceX did?

Blue Origin has a rocket that can go up to 100km and come back down. SpaceX's Falcon 9 is an orbital rocket which is a much, much more difficult achievement. But only the first stage of the Falcon 9 is reusable, with the second stage begin lost on each launch. The Starship rocket under development is being designed to be fully reusable with a quick turn around like an airplane. In fact SpaceX hopes to see Starship used for quick intercontinental trips on Earth.

Blue Origin is building a large re-usable rocket like the Starship called the New Glenn, but its first launch date keeps slipping. I would guess that Starship will be launching first but it is really cool that two groups are working on such rockets.

The boring company is a bet on improving tunneling machines for speed and cost. This might not be realized but there's no good argument that proves it to be impossible.

There are fields where physical limits were already reached, for example rocket engines specific impulse has barely improved since the 60s because of fundamental chemistry, but this is not the case for tunnels.

Provided enough funding it should be possible to dig faster, and cheaper per km.

Its quite funny to me that literally everybody that is against Boring company is sighting these 'Thunderf00t' videos, and judging from some of his other videos he has a pretty clear bias. He is also a chemist and why his knowledge of engineering is better then those of the engineers at SpaceX and Elon Musk is highly questionable to me.

> the safety issues of a car breaking down in a tunnel

The cars have front radar and can break. It would stop the line but I don't see how it is a safety issue. Most danger in tunnels is from bidirectional traffic that Boring avoids.

> the enormous cost with digging deeper tunnels

The same could have been said about other Elon Musk projects. Nobody is surprised that an electric car can have better performance and power and yet a diesel driven tunnel boring machine can not be improved? Tunneling is incredibly conservative and uses both diesel driven tunnel boring and diesel driven locomotives to transport out the dirt. The inherent problems of having lots of diesel engines a tunnel are quite clear and eliminating it has clear operational advantages.

> underground road tunnels are just not economical or particularly safe

What the evidence for underground one-way tunnels not being safe?

> These kind of infrastructure problems are just hard and have a ton of trade offs, there is no silver bullet.

Therefore we should try to improve it with modern technology from other fields? Its not a silver bullet, it is tacking new technology from automotive and aerospace and applying it to tunneling and mass transportation.

> a diesel driven tunnel boring machine

!? I'm not a tunnel building specialist, but a cursory search seems to indicate that tunnel boring machines commonly already are electrically powered by simply using a very long high voltage cable stretching all the way from the tunnel entrance. Whether that cable then connects to the electricity grid, or to a bunch of temporary diesel generators stationed outside of the tunnel is a different matter, which in any case doesn't bear on the tunnel boring itself.

>I really wish Elon’s boring company were further along

It's unfathomable to me that people believe car tunnels are the future of anything.

Right now The Boring Company is trying to recreate an underground train in Las Vegas, and is already behind schedule and running into issues people predicted before this started:

https://www.theverge.com/2020/10/16/21519692/elon-musk-borin...

Agree. Personal rapid transit system might work but cars, nope!

What the difference between electric buses behind each other in a tunnel compared to multiple rail way cars behind each other?

One system is cheaper because the units are mass produced far more often and the system is more flexible.

Carrying around a battery rather then having complex high power electronics along the tunnel seems like a no brainier as well.

Las Vegas is literally a first of its kind system, being a few month behind is not surprising or concerning. And the problems have to do with specific fire codes, not any inherent flaw in the system.

It's been stated elsewhere in this thread, but there really are a lot of practical, well-understood engineering issues with Elon's proposals that he refuses to admit or deal with. Here's a somewhat humorous review of his claims / what's wrong with them and how they stack up to actual civil engineering projects:

https://www.youtube.com/watch?v=4dn6ZVpJLxs

It's actually very interesting .. humans/lifeforms are wired to deal with critical irreversible errors.

Reply on news.ycombinator.com