Hacker News Reader

Comment by mabbo | Hacker News Reader

I think the problem we have with self-driving cars is more social than technological at this point.

Consider the hypothetical: a million self-driving cars on the road that, collectively, will have 1/10th of the fatal accidents that human drivers would have[0]. But, the ones they do have are accidents a human driver would almost certainly have avoided.

Is this something we would accept?

My guess is that no, we wouldn't. Because the accidents avoided don't make the news, but the accidents that occur- especially ones that you say "my god, how did it screw that up?" will make the news, and our perception would be that they are more dangerous.

Until Waymo's cars are better than most humans in every single situation, they won't be able to win over the public perception war.

[0]I'm making those numbers up. I acknowledge that. But it's a hypothetical so give me some leeway on this!

View on HN

You're hitting the nail spot on with this one for me. As a blind pedestrian, I very much feel I am in danger of falling into exactly that group of potential victims you are hinting at. Right now, I have the illusive comfort of the "Vertrauensgrundsaatz" which basically tells every driver obtaining a drivers licence that they need to take special care when it comes to disabled pedestrians. Sure, one might say these new self driving systems will "just" have to follow that same rule as well, but I am very much doubtful this is technically possible. So currently, I feel like the drive to put innovation on the streets and pull money out of pockets is actually actively endangering me in the future. Not very bright outlook I must say.

By your use of German, I assume that you experience a far higher driving standard than people in North America and frankly most of the world, and even that is far from perfect.

The Vertrauensgrundsatz does not account for distracted, tired and inebriated drivers.

My guess is that self driving cars are already far, far better than human drivers at not hitting pedestrians.

My guess is that it depends a lot on the driver. There are a lot of really reckless drivers, or really bad drivers out there on the roads right now. I know people who've been in several (relatively minor) car crashes, driving the same-ish sort of route to/from college and work that I was during the same period. I think they're just a bad driver. I feel like they're driving recklessly when I'm a passenger in a car they're driving. So a self-driving car might be a better driver than them, while also being worse than the median driver, and much much worse than like the upper portion of the human driver spectrum, like the 1st quintile of the 20% of human drivers who are pretty good.

Given how reckless some drivers are, I'd imagine the distribution of crashes among drivers is very skewed. I don't know numbers, but I'd expect to see something like 80% of car crashes involving (or objectively caused by) 20% of the drivers, with most car crashes involving people who have been involved in numerous crashes, and a large minority of drivers having been involved in zero car crashes.

> My guess is that it depends a lot on the driver.

There is a lot of variation in drivers. Young ones with good reflexes and senses but bad on experience, old ones with worse reflexes and senses but better on experience. Ones that stop at stop signs, ones that blow through them if they think they can get away with it. Cities like New Orleans where drivers are much more aggressive because the cops don't care, etc...

I think the important point is that self driving cars are likely better than the averaged real world population of existing human drivers at avoiding pedestrians.

The average is not the only stat though, and doesn't tell everything. Even if it were true that FSD was safer than the average of human drivers, replacing some manually-steered cars with FSD cars could result in more pedestrians getting hit, depending on the breakdown within the two groups. The overall average isn't enough information and obscures the internal distribution of skill/safety within the overall population of drivers.

It matters who is driving what cars. If you're taking safe drivers and having them drive FSD cars, and comparing the average of that against the average of all non-FSD driven cars (which includes a lot of really shitty drivers in the average), then the net safety impact of this could be nothing. Introducing FSD cars would have no impact to overall car safety, and might even have a negative impact, if the FSD software is worse than the good drivers who are now not driving manually-steered cars but are driving FSD cars instead.

You might think that the average is not too terrible, but wow, there are some people at that one pointy end of the bell curve that make you shake your head.

I present, "Canada's Worst Driver," a series that I watched quite a bit when I had a cable subscription. Besides being hilarious, it should make you think a time or three before crossing the street. https://www.youtube.com/playlist?list=PLHB4WuM7Y_eP-PFv5F6To...

The issue i speculate is the tradeoffs are more imposing than with a human. Sure i accept that the car can drive more safely than a human (all it has to do is never exceed 1mph) but as safety increases i speculate it involves accepting tradeoffs like cars getting stuck, driving slowly, blocking traffic, hard braking when not necessary, or even restricting the operating domain.

As a pedestrian who isn't disabled but just inattentive and erratic, I know I will feel much safer around 100 waymo cars than 100 human cars.

The crucial word in your sentence is "feel". If you think technology can only do good, you will likely feel safer around AVs. And since I have an inherent distrust regarding complicated tech, I "feel" less safe with more automation. Both of our feelings are subjective. The question is, will we be less or more safe?

Depends on the safety statistics of the specific tech. Around Waymo? Safer!

I wonder if we’ll all start wearing some kind of markers that these cars can more easily pick up. Like a blind person’s cane could be a specific hue or be otherwise easy to detect by driverless cars. Or cars could detect the UWB signal (like AirTags use) from our phones which is very precise.

Basically, maybe there’s better ways than standard vision (which for computers is still kinda basic compared to a person) to solve the pedestrian hitting problem.

Yeah but what a sad dystopia where you need to wear a beacon just to be outside without risking death, especially if like jogging on a sidewalk or riding a bike.

The cars are clearly the danger to everyone who isn’t in a car, why would the burden be on everyone else, all the time?

To be completely fair: joggers and bicyclists already wear such beacons.

I really don't want to be fair though, because I think your point is totally correct.

Why?

Because reality.

Since the advent of motor vehicles, I would argue risk has generally scaled with the efficiency and power of vehicles.

At the minimum, for 70 years we have known that danger to both pedestrians and safe drivers is ever present and growing. The reality is not about who SHOULD deal with this problem, but how to most effectively arrive at a functional solution which actually reduces risk for drivers and pedestrians alike.

As well, it's worth noting most accidents occur in areas where you drive most often and you're most familiar. And the increase in traffic accidents and fatalities over the last 10+ years has been due to driver distractions, ie cell phones etc, more than anything by a lot.

Yes there are reckless drivers out there doing stupid shit. However, that's always been the case. Now it's just easier than ever to capture them on camera and for them to go faster than ever for cheap. However, even back in the 70's, crash safety technology was laughably behind anything we have so it goes both ways imo.

> As well, it's worth noting most accidents occur in areas where you drive most often

It would be more remarkable if most accidents occurred in areas where you rarely go.

Where I am from, there used to be a time when certain people had to wear markers. We dont want to be reminded of that.

On the other hand, you might be able to let self driving car take you anywhere instead of asking someone give you a ride.

I don’t think a self-driving hearse will calm their nerves

People showing up to their destinations dead, having passed away in a self-driving vehicle, is inevitable. May as well make it work on our terms.

I seldomly ask others for a ride, I prefer to pay a taxi cab. And no, I would still prefer a taxi driver over an autonomous vehicle even if they were available, because a human is still very much useful when it comes to the last 50 meters of a trip. I often ask for guidance to find the correct entrance, for instance. BTW, your argument is very cheap, I tend to hear it often. "You are afraid of roboters? Here is one, play with it, you will like it!"

Like my driving teacher told me in lessons (to admonish my safety focused thinking): do you really think everyone you're sharing the road with has a driving license? I guess he has a point. There are people driving out there without a license, or with suspended license, and all kinds of various situations. Which is why we drive defensively and have to be ready for drivers that don't always follow the rules.

I would argue that someone without a driving license will drive more carefully not to gez caught. Also suspended license doesn't make you forget the rules.

> Also suspended license doesn't make you forget the rules.

No, but there's probably a good reason it's suspended. It's not that you forgot the rules. It's just that you're not following them.

You can get your driver's license suspended for all kinds of driving-unrelated stuff, including not paying child support (which is what I don't get at all - how is someone without a driver's license supposed to go to work to make the money?!).

"how is someone without a driver's license supposed to go to work to make the money?!"

Presumably by paying their child support.

I don't know if it works as intended, and it wouldn't surprise me if people have horror stories about clueless rulings by clueless judges, but obviously the intent is that the license is suspended because the person is intentionally not paying child support.

> > Also suspended license doesn't make you forget the rules.

> No, but there's probably a good reason it's suspended.

Not paying the renewal fee on time is a big reason. That's a rule, indeed, but nothing related to driving or safety.

That seems rather naive. Most people driving without a drivers license probably aren’t thinking that rationally…

Self-driving cars should be far better at recognizing a human in most orientations/poses/outfits, regardless of the car's angle, or weather. It doesn't get distracted, can see 360 around the vehicle, has faster reaction, and is specifically timid around pedestrians. Even the best drivers could easily hit someone based on conditions.

There have been suggestions that pedestrians could make cities interesting, since a place like New York would have every pedestrian no longer afraid to just walk into traffic. The cars will just wait until a fully clear moment that may never come

It's this adversarial environment that I fear is the complication.

As you say, if you know the cars will stop and let you go, this power will corrupt.

Extend this thinking to bicyclists, skateboarders, delivery riders, motorcyclists, car enthusiasts driving themselves - all able to simply cut off a self driving car knowing it will give way.

But there is more adversarial out there.

AI input hacking to get known responses. Cars disabled because of gang tagging on sensors.

Car enthusiasts or privacy enthusiasts (privacy shouldn't require enthusiasm but here we are...) updating car firmware/software. People with full physical access!

And we haven't touched on the ethical stuff which is a world of discussion in itself.

Self driving cars by evolution in current societal rules etc will take a LONG time.

> if you know the cars will stop and let you go, this power will corrupt.

> Extend this thinking to ... - all able to simply cut off a self driving car knowing it will give way.

But this is already the case; if I jump into slow moving traffic in a (western) city centre the cars will let me through, possibly even without honking. The only things to fear are distracted drivers (which is the same with the AI, because I can't know whether the AI sees me) and the occasional madman, but those are rare.

What's more interesting as you point out are situations where other drivers will take advantage of the AI's adherence to rules and safety. That's going to be especially fun in cities like Napoli, where things like driving through red signal are completely normal. It's probably not going to work there at all in its current state.

>As you say, if you know the cars will stop and let you go, this power will corrupt.

Would stepping out in front of a car be illegal?

Which, in the abstract, is an interesting theoretical question, but California recently decided to basically legalize jaywalking.

https://laist.com/news/transportation/understanding-new-cali...

That's the thing isn't it.

If "person walks in front of car" event is rare, then the automatic car stop saving a life or minimising injury is nice. Saved a person from a bad day for a minor inconvenience for the car people.

If a NY city worth of people do it without regard, then cars will be gridlocked.

My guess is:

- Assuming self driving cars that that stop for pedestrians are common

- then laws against deliberately walking in front will exist

- to judge the dispute of "deliberate" there will be more cameras, because why wouldn't there be...

It's long past time that we turn the streets back over to pedestrians. Communities are much more enjoyable when car-free.

Potentially, but sometimes I think we do forget how surprisingly good humans can be at this, and how hard it can be for computer detection systems to work well enough without also having a high number of false-positives where it will slam on the brakes for something it didn’t need to stop for (and in doing so, potentially even causing an accident!)

> [...], since a place like New York would have every pedestrian no longer afraid to just walk into traffic.

London already has near suicidal pedestrians.

And I say that with respect: it's the opposite of the usual tragedy of the commons. Every pedestrian in central London who just crosses the street without looking places themselves in appreciable danger, but contributes a tiny bit to training London drivers to be extra careful.

(Going off on a tangent: tipping is an interesting opposite.

If everyone in the US stopped tipping tomorrow, economic equilibrium would shift so that service sector people would have comparatively higher nominal wages and roughly the same overall income. That's how it is in Japan or Australia or Singapore (or the US before the 20th century), where tipping is either less common or severely frowned upon.

Because eg prospective waitstaff in the US expect to be tipped, they accept lower wages. Similarly, a single patron not tipping in the US gets a lot of social awkwardness without shifting the equilibrium much. A single waiter in the US not accepting tips loses out on income and also gets social awkwardness, also without shifting the equilibrium much.)

>Sure, one might say these new self driving systems will "just" have to follow that same rule as well,

Next round of CAPTCHA: click the squares with blind pedestrians

think about it, this is how these things are being trained--by people frantically trying to click stupid images just to get to the webpage they wanted.

This is one reason I never understood Tesla's vision-based approach. In order to be accepted, self-driving cars don't need to be just somewhat better than humans most of the time. They need to vastly better in every situation, as you mention, to the point that they'll need every sensory advantage they can get.

I got out of a ticket once because I didn't see a "no through traffic" sign against a bright sunset. No chance that same cop gives a self-driving car a pass, nor should he.

Karpathy claimed they worked for years and could not reap the benefits from multiple sensors however hard they tried. He seemed really convinced and does not get to me as one who tells stuff just to justify cost reductions, like Elon sometimes is carried away.

It’s also possible they weren’t able to see benefits given the processors that were available to them at the time.

It’s also possible that they were so far behind Waymo in the journey to FSD that they weren’t yet at a point where multiple sensors would make a significant difference.

This is my read on it as well. I listened to the same interview (it was definitely on the front page, if not at #1 for a while). I stepped away feeling like Karpathy had described a lot of good business reasons for not using other sensors, but not a lot of good technical reasons. Sensor fusion is hard, yes, but maybe not harder than perfectly re-projecting 2D pixel images into 3D vector space.

Just my interpretation, but it felt to me like a hail mary because they were fully committed to being the first mover. Waiting around for LiDAR prices to come down would have meant that Waymo would have beat them.

I always thought it came down to two reason, costs and looks. Telsa has to sell a car people want to drive daily. They can't have a bunch of lidar sensors on their cars no one would buy them even if they could drive themselves. Also they would most likely cost a lot more due to lidar sensors not being cheap compared to normal cameras.

Unlike Waymo who doesn't care about selling cars to people to drive daily. No one going to care that the taxi they are taking looks ugly as long as it get them to the place they are going for cheaper. The cost is also a less of a factor due to them being able to produce an income by charging people to ride in them.

> They can't have a bunch of lidar sensors on their cars

Why not? There are production cars with lidar now that isn't the big spinning thing on top of the car.

Then something else was the bottleneck at the time. It is very easy to prove that some sensors in some situations will be able to perceive things that other sensors cannot. In those situations the additional sensors are crucial first steps. I would guess the bottleneck is shitty reliance on statistical machine learning with a long tail of unhandled edge cases. Each case very uncommon, but in aggregate a very important sum.

You are misunderstanding the situation. Karpathy claimed that sensor fusion of radar, sonar and vision isn’t working well. He made no such claim about Lidar. Lidar is the sensor that is the crucial difference between Waymo and Tesla's approach to self driving.

The reason he claimed sensor fusion was not working well was due to vendor versioning. He claimed the same sensor from different manufacturing batches performed differently and thus needed to be re-characterized, which then has follow on effects in various math models. Multiply this by many sensors, and the need for replacement parts inventory for a decade or two and the problem becomes intractable was his claim on his most recent appearance on the LF podcast.

Almost sounds like a supply chain/manufacturing problem than a software problem.

Wasn't the Summon feature downgraded when they moved to a vision-only approach? One YT video I saw compared a version 1 and a more recent version and the vision-only wasn't able to do as much. Contradicts the idea additional sensors do not add value.

I think the idea is less "additional sensors don't add value in various scenarios" and more "we are 100% certain a vision-only system can perform well on existing road infrastructure; we are not sure about sensor fusion systems".

The problem is, in reality, we have Tesla vehicles that sold with other sensors that are now disabled, and those cars are now very difficult to drive with any automation enabled, even simple cruise control, due to the limitations of vision-only driving. And this is leading to them slamming on brakes at inopportune times, blinding oncoming drivers with the bright headlights, etc. - issues that did not happen when other sensors were used.

In reality, we have Tesla vehicles that sold with other sensors that are now disabled, and those cars are now better to drive with automation enabled than with the extraneous sensors, even simple cruise control is better and phantom braking is actually decreased vs spurious radar returns that existed previously, due to the limitations of low resolution radar. Auto high beams is now better at not blinding oncoming drivers with bright headlights, etc. - issues that happened all the time when other sensors were used previously.

How is your experience the direct opposite of the parent post, given the same change?

Believe it or not, if you spend enough time with a Tesla, you quickly realize that actually detecting things is a solved problem.

The thing they need to improve and are doing so rapidly is actual trajectory policy calculations.

And that’s not going to get better with more sensors seeing the same things it already sees.

This claim is not true.

As someone who drives a Tesla with the FSD beta, the vehicle has been getting progressively better since 2018.

It’s drives smoother and brakes more predictably dive they stopped using the front radar.

This claim is not true.

As someone who owned a 2019 Tesla and who owns a 2023 Tesla, the older car had better autopilot. It was starting to degrade in 2020. The new one is worse. Phantom braking was quite rare in 2019. The very first night I drove home the new car, it phantom braked on a lonely, empty stretch of I-5.

I want the radar back.

FSD stack is disabled on highways. You are using the years old code. Beta v11 when it comes out will enable OP's FSD referenced improvements for highways.

I used to mock my relatives for not wanting to put their kids in a Tesla. But yeah, I'm not subjecting my family to beta software for a critical safety feature, nor should anyone else.

I definitely have some relatives that I would trust less than Tesla to drive with my kids.

If you're referring to the Lex Friedman interview, at the start of his answer, he mentions it was a cost-based decision. And that radar/ultrasonic wasn't worth it for them, due to the additional time it took. Not that it wasn't helpful, just more effort that could be better spent elsewhere.

He clearly states that extra sensors "contribute noise and entropy into everything. And they bloat stuff." https://www.youtube.com/watch?v=_W1JBAfV4Io

Essentially that trying to utilize multiple sensors cripples any progress (given that resources will never be infinite).

It's too bad that none of their competitors, who all have better results, disagree.

While technically true that extra sensors contribute noise, surely with the proper programming they should be helpful. You just need to weight the information from you extra sensors appropriately based on your confidence the signal is accurate.

For example, if your lidar sensor is 99.999% percent sure there is an obstacle in front of you, surely it's helpful to take that information into account, even if it is a tiny bit uncertain/noisy.

Wouldn't that be more related to parallel processing power and throughout capabilities than feasibility?

More data in any situation where bandwidth is already maximized will lead to entropy and noise. However, if the capabilities were there to process all of that data in low latency scenarios with headroom to spare, surely adding additional sensors and data points would lead to a more complete model of spatial awareness for the car.

That's all hypothetical and reliant on ignoring the realities of running a business and tech development lol

A bit like our discussion here. More standpoints can ideally be merged into a coherent, more complete picture. The key is to sort out disagreements as coming from different backgrounds and biases and everybody admitting she is not 100% right. Otherwise it is a quarrel of stubborn knowitalls that can't agree.

Fusion is hard. As hard as getting humans to agree. Been there. In both situations.

And of course you can concentrate on improving one echo chamber, err single sensor. But you can never come past its fundamental limitations.

I also never understood why we had to use "vision" approaches that have the same visual spectrum as what humans see. Any sort of sensor on a device is already synthetic, why limit the spectrum that you attach it to? Should use light sensors, sound sensors, gps, everything.

So far nobody really cared how road markings, signs etc look outside of the human visual spectrum, so there's likely a lot more variation there. Both in terms of how things look like when new, and in terms of acceptable wear.

Right. But that is in designing the existing roads. I'm talking about the cars. And I'm specifically asking why not adding more options? I don't mind a camera being part of the solution at all. Gives an obvious path to human labeling of training data. But, why not have more?

We have a good idea how all this stuff looks in the infrared spectrum, and that could help a lot in the sensor fusion.

And even then, the best cameras today are still a long ways off from matching human eyesight. Tesla's cameras are not state of the art, either.

I wouldn't be shocked if the best cameras are better. :)

Point stands that the ones being put in the cars aren't.

The best cameras have much lower dynamic range than eyes (particularly in low light), and much longer times to refocus. Not to mention higher feedback latency when focusing on a particular object: eyes and brain are very tightly connected.

The eye is an incredible piece of machinery, specifically for dynamic processing like this.

I meant that mostly tongue in cheek. That the best camera ever produced may be better than our eyes. I wouldn't be shocked if such a thing exists. Probably target large and would be way more than a single sensor camera, though.

And agreed that eyes are impressive.

Right so to approximate the capability of human eyes you would need multiple coaxial cameras with varying apertures. Then stitch the images together in software.

One of my biggest concerns about Tesla's vision based approach is that it appears to be entirely about cutting costs. Nothing about it says they actually think it is superior; the cameras on a Model 3/Y are mediocre. They were mediocre the day the Model 3 was first released. If you were going to rely on a vision system in a serious way, you'd at least invest in better camera tech. Hell, Subaru EyeSight has a significantly better camera setup, last I checked, and who looks to Subaru as a technical leader?

Someone else said it here on HN, and I think they're absolutely right -- Tesla is all about vertical integration, and this is preventing them from excelling at anything other than saving pennies. A good part of why the new EV competition is doing everything better is they didn't roll their own tech. They bought packaged solutions from companies that only do one thing, but do it well.

have you driven a tesla autopilot at night in heavy rain with no visibility of the lanes? I did, I was amazed at how good it was. I couldn't see lanes but it did and judging by the picture on the screen it saw them very well. More pixels doesn't mean better, same as more ram doesnt mean better (iOS example).

Yes, I'm on my second Tesla in fact.

I have even earned the dubious honor of having been pulled over for suspected drunk driving because I used AP on a trip home from my MIL's house after Thanksgiving dinner. I was amused by how much trouble AP was having with mild corners on I-5, the state trooper behind me was less amused. Well, at least until he established that I was sober, then we had a good laugh about autopilot and he gave my kids sticker badges. A happy ending for everyone but I didn't activate AP again that night.

Maybe the problem was that it was clear and dry? Ya think it would have been better with some road spray on the cameras?

Subsequent to that, I ended up abandoning AP altogether because it was so jarring when it would decide an overpass looked like something it should slow down for. My wife suggested that she'd be happier if I just used my foot for the accelerator. For some reason we still ended up buying another one, but after the phantom braking on day one (empty freeway, overpass, apparently this is very much now "a thing"), I guess I'll just skip AP before even starting. I wonder if I could ask Tesla to deactivate AP altogether so I could have traditional cruise control?

Sounds like Tesla should update their autopilot software to force itself to be enabled and prevent driver control at night when it’s pouring rain.

FWIW, the Subaru system is fantastic — I use it daily and it’s very predictable and literally never been dangerous. And works really well in stop-and-go traffic!

It’s not FSD, but if you use it as a driver assist, it takes a giant amount of burden off of driving in traffic. Even on a busy highway that’s going somewhat fast, it’ll just follow the car in front of you. And truly, it’s never once been dangerous.

Only catch is that it can’t “see” very long distance — if you’re going 70mph and there’s a stopped car in your lane, it probably won’t stop (at least, I wouldn’t test it!). But again, it’s driver-assist and not FSD, so you’re still expected to notice that. And within its limitations, it is really exceptional.

The Subaru EyeSight driver assist system is pretty good. But I have seen it suddenly deactivate itself several times in heavy precipitation.

On the contrary, you can hear Andrej Karpathy explain how other sensors produce noise & are too low bandwidth when compared to vision. Other full self driving systems will probably follow suit & get rid of impossible to calibrate lidar systems. How are the cameras mediocre? Do you want even higher resolution cameras? This would cause compute turmoil. https://www.youtube.com/watch?v=_W1JBAfV4Io

They are standing firm on the vision-only approach because it is the correct approach. FSD cannot be perfected unless the Tesla team puts 100% exclusive focus on perfecting vision based models that don't have lidar as a fallback. Tesla can only use lidar again for redundancy only after vision is fully solved problem.

https://tvtropes.org/pmwiki/pmwiki.php/Main/BurningTheShips

Combing multiple sensory inputs is a hard problem in itself. Say Tesla uses LIDAR with vision...they can have a great vision experience, a great LIDAR experience, but fuses those two experiences together can create lots of problems that wind up being worse than either by itself.

So the beatings will continue until morale improves, eh?

This is like saying we need to get to the moon using steam power and ONLY THEN we can improve by using rockets. What if it's completely infeasible to solely use vision for FSD?

Even humans use audio processing to augment the vision.

Tesla May Be Adding A Radar

https://www.forbes.com/sites/bradtempleton/2022/12/12/tesla-...

Archived: https://archive.vn/20221212143745/https://www.forbes.com/sit...

I agree with you entirely.

Continuing with your hypothetical, even though we’d be 90% safer as a collective, the safety of the individual feels compromised: the risk of an accident is non-uniform when involving humans (depending on e.g. age, experience, safety, alertness, etc.), but becomes uniform (or at least more uniform) with an algorithm in charge.

That’s a tough thing for people to buy into.

A very good observation. Based on your comment, I think we can relax the requirement stated by OP by saying:

"Until Waymo's cars reduce any individual's chance of an accident."

So for example, suppose a Waymo car is better than humans overall, but tends to do worse than humans when there's a small bump on the road. And suppose that all humans (in a given regulator's area, e.g., California) tend to encounter such bumps at roughly the same rate (per mile driven) over their lifetime. In that case, it's probably going to be acceptable, since every individual is better off.

I don't know, maybe this is not impactful / obvious enough for people to care about?

What certainly is obvious is that the safest drivers are much safer than an average driver (does anyone know of a study that estimates this ratio?). Therefore, at the very least, the threshold for Waymo should be not the average accident rate, but the accident rate for the safest drivers.

> the safety of the individual feels compromised

Exactly. I have had zero accidents in 20 years; I'm not interested in a car that will lower the overall accident rate if it increases mine.

Even the most perfect defensive driving won't fully mitigate the unsafe driving of others though. I'd rather have them to be in autonomous vehicles.

Sure, but nobody thinks they are the bad drivers.

> I'd rather have them to be in autonomous vehicles.

Who is "them" and who allocates the groups?

We can't even get any kind of car control skill testing or training to be part of a driver license issuance.

I think you're mistaken. There are still massive technological challenges. I have seen nearly no evidence that current self-driving car technology is even remotely close to matching the ability of a novice human driver. Sure while the "don't crash into things" algorithms may generally be fine, these systems seem to frequently deadlock in completely mundane situations. They also seem dependent on remote operator assistance when encountering non-ideal conditions, greatly limiting their maximum speed.

If anything, legislation and social acceptance has moved faster than the technology. That's the opposite what many of us observing this space expected 10 years ago.

At this point I'm starting to have doubts about whether the full dream of self-driving cars will even be realized within my lifetime.

I read this comment after taking a cruise in SF, which is a self driving cab with no driver. It basically reminds me of all the comments saying that VR has no future, written by people who have never tried VR and would get their mind blown by it if they tried the latest iteration. Maybe you should come to SF and try one of these self driving cars yourself :)

I actually do live in the Bay Area and spend a lot of time in San Francisco. I applied for the Cruise waitlist well over a year ago but have not been accepted. I've tried to organize with friends who have access but we rarely have a reason to go the Richmond or Golden Gate Park after 10PM. The coverage area is very limited.

I'm impressed that they're actually offering driverless rides on SF streets, but my point stands. The cars operate only on the slowest streets at the quietest hours. Any problem they encounter is handled by remote operators.

I'm not outright dismissive of self-driving cars. I truly want them to exist. I don't even own a car and dislike being behind the wheel. I just don't buy into infinite hype pushed by a revolving door of charlatans.

Also I do have a modern VR headset and celebrate the technology. But, to make a similar comparison, the metaverse "ready player one" vision is not within our lifetimes.

They don’t operate only on the slow streets. They operate mostly on a large rectangle in the north of SF (basically where I live so perfect for me).

Funny, because VR has no future outside of some niche areas. It's the new 3D TV. The average person doesn't want a computer strapped to their head all day.

I ALWAYS thought 3D tv was the biggest gimmick. I was the first to point it out. And I’ll be the first to tell you that VR is freaking mind blowing and your comment is going to poorly age.

How does having taken a ride make you an expert? That's a ridiculous argument on a level with trying to shut someone who criticizes Facebook's algorithms down by saying I use Facebook and never had an issue.

If you're into self driving tech, Cruise especially is regularly ridiculed for their vehicles blocking traffic. Take this latest example: https://twitter.com/as611/status/1597144790767788032

But more importantly in case you're not aware it's not true self-driving at all, they're geofenced and both Waymo as well as Cruise have been shown to have issues with even minor changes like a construction site. The tech isn't there, we don't really have full self-driving and if we actually put a million "self-driving" cars on the road as the above user suggested, it would be utter mayhem with current tech. If this will change in the future is to be seen and from what I've observed progress has already slowed as per ninety–ninety rule. It's easy to have some autonomy functions. It's hard to have millions of fully automated cars. So far, we're nowhere near it.

You might be right, but have you seen this: https://old.reddit.com/r/electricvehicles/comments/z9b1rq/wa...

This level of sophistication makes me think it will not "frequently deadlock".

A bigger problem is this: Say you need to prove to the public that the autonomous car is significantly safer, and you do an apples-to-apples comparison between a hypothetical Level 4/5 car and well designed new Level 2 electric car like a Volvo C40 or a BMW i4.

The modern Level 2 car is already today at below 1 fatality per billion vehicle miles travelled (VMT). The autonomous car then needs to be below 0.1 fatalities per billion VMT. Meaning that if you have 1 million vehicles of your make deployed, they each need to have driven 30-40 000 miles autonomously before you have enough statistics!

That means proving the safety of a Level 4/5 autonomous system is extremely expensive and slow, and requires significant public adoption before it's proven to be safe. The consequence is that, assuming proven safety is necessary before public adoption, it becomes impossible to prove safety.

Another point is that OTA upgrades for autonomy become entirely pointless, as you'll be polluting your statistics if you change the code more frequently than every ~3 years.

I think you’d use crashes, not fatalities.

> The modern Level 2 car is already today at below 1 fatality per billion vehicle miles travelled (VMT)

Does this statistic count fatalities to people outside the car? I ask because (in the US, at least) car safety ratings don't take those into consideration.

> in the US, at least) car safety ratings don't take those into consideration

Which ratings? The NHTSA does include non-occupant fatalities in their statistics. E.g. https://www.nhtsa.gov/press-releases/2020-traffic-crash-data...

NHTSA safety ratings (https://www.nhtsa.gov/ratings) only count people in the car. It looks like they proposed a change to that this spring (https://www.nhtsa.gov/press-releases/five-star-safety-rating...), but I don't think they've implemented it yet. From that press release:

> For the first time ever, [the proposed rating program] includes technology recommendations not only for drivers and passengers but for road users outside the vehicle, like pedestrians.

News story on this: https://www.theverge.com/2022/3/3/22960262/nhtsa-ncap-five-s...

To me, self-driving seems like a band-aid fix for terrible car infrastructure. Driving a car is already one of the most dangerous things the average American can do. Due in part to larger, heavier vehicles, high speeds in residential areas, etc. Even if you had a "perfect" driver that doesn't prevent someone from ramming into you.

> one of the most dangerous things the average American can do

About comparable to the risk of falling. Lower risk than suicide. Or death by opioid overdose. And of course, the most dangerous thing most Americans do, by a huge huge huge margin, is overeat and lounge on the couch.

Imagine thinking a picture of a penis is sexual harassment...

I agree it's a social problem, but IMHO it's a rather different social problem: Current cars, roads and car culture is adapted to human drivers, and AI is expected to be able to integrate into that.

What if we would make cars and rules that are adapted to AI cars and ignore human drivers? e.g. Ban human drivers from some roads, allow AI cars with designs that exploit AI advantages (e.g. much better reaction time) but do not require or even allow human backup (enabling us to put the passengers in a secured shell), etc. I suspect we could than reach a 1/20th rate today.

Why self-driving trains are much easier to implement, yet there are not that many systems capable of doing that? Many newer metro lines are GoA 2 or 3, theoretically capable of running autonomously, but they always require a driver in the loop.

My partial answer is, making an extremely reliable system is hard. If someone wrote a deadly bug even only happen at a very corner case, it still can kill people. And it's quite hard to prove there isn't such bugs.

Regulations. Nobody ever got fired for buying microsoft or for requiring a driver in the seat.

We don’t half self driving trains because there’s far, far less incentive.

You only need 1 or 2 drivers for a huge train carrying a lot of ppl/stuff. Optimizing away the driving barely reduces the cost of operating the train as a whole.

With cars, everyone drives themselves. There’s a lot more driving happening and so if you can automate that away, you create more value.

> Until Waymo's cars are better than most humans in every single situation, they won't be able to win over the public perception war.

The current situation as basically the opposite: waymo's cars are better than humans in almost zero situations. It's hard to gain the my trust when your car can barely drive in a drizzle.

I agree.

> But, the ones they do have are accidents a human driver would almost certainly have avoided.

I suspect most human-driver accidents are also accidents that (other) human drivers almost certainly would have avoided.

That's scant consolation for all the people dying in traffic accidents each day, of course.

It is a valid point but the financial incentives are so big that some jurisdictions will allow it. In fact they already do allow these autonomous systems on public roads. That is going to continue to expand and since the financial incentives are huge even when deaths happen the governments will continue to allow it.

And in fact some regulators fully understand the tradeoffs and will prefer autonomy for the better good of the public. An example of this is the Boeing 737 Max, those crashes wouldn't have happened if there were no autopilot systems. But regulators are not suggesting that all autonomous systems on planes be turned off because of the safety and financial advantages of keeping them in place even though they are obviously not perfect.

> Boeing 737 Max, those crashes wouldn't have happened if there were no autopilot systems.

Bad example. MCAS was an obvious case of a criminal corporate behavior, not a tradeoff between overall safety vs. technical perfection.

It doesn't matter, the point is that relying on autopilot will open up crash risks that do not exist with manual flight controls

It does matter! I'm assuming that any sane person, especially a highly trained professional, is not fully relying on automation and has at least some idea of its limitations. In the case of MCAS, corporate fraud distorted the level of trust pilots had in automation, leading to loss of life.

I think this is more true than not. But also underestimates the technical problem.

Airplanes are not fully autonomous, even in the instrument flight rules system which is highly standardized. You don't have non-standard or non-predictable things. In instrument conditions, only instrument planes and pilots exist. While portions of segments are automated, transitions between segments and phases of flight aren't. It's ripe for automation, yet isn't fully automated.

The automobile environment has more objects thus more density of complexity, more non-standard and non-predictable actors like non-autonomous vehicles along with bicycles, mopeds, pedestrians, etc.

Yup, exactly. "There was a 1% national accident rate before autopilot but now it is 0.5%, aren't things great?" Not really, because my personal accident risk just went up from 0.1% to 0.5%.

But you have no issues with paying for social insurance when you never use it yourself? Isn't this the same kind of tradeoff?

How is it the same tradeoff? I'm not increasing my chances of getting sick by buying insurance. The worst case is that I won't use it, which is fine. The worst case for autopilot is that I'll get into a crash because the camera couldn't see a massive truck in front of me because of glare and shoddy software.

It is not only a social, but a legal issue, too. If a human kills someone in an easily avoidable accident, breaking the traffic laws, he may go to jail and/or lose his driving privileges for a while. If an artificial neural network does the same, should it loose driving privileges for a while? Should someone go to jail?

To double down further on the social side of this, public transit is largely getting there faster than point to point driving. In that many trains and such are already largely "hands off the wheels" for operation.

Relatedly, another "problem" with "self-driving" cars is that we want all of the convenience and ease of use, without adjusting liability and ownership considerations. Consider, if Waymo gets to the point where they have a self driving car that you have to have a subscription to use, do you own the car? Are you the liable for any accidents it has?

To lean in on that hypothetical. I'd imagine a lot of families will use self driving cars to send kids to school. Is effectively a bus that terminates at your house. Who is liable for a mistake if the operation of it is completely remote?

> Consider, if Waymo gets to the point where they have a self driving car that you have to have a subscription to use, do you own the car? Are you the liable for any accidents it has?

I'd imagine the company would take on liability, as long as humans can't drive the vehicle or they aren't driving when the accident occurred. Mercedes already got the ball rolling on this [1].

[1] https://www.roadandtrack.com/news/a39481699/what-happens-if-...

I can't see that path getting taken without you basically losing ownership of the car?

Agreed, but I can see it being solely a "lease" model. For companies to take on liability, they'd have to make sure a customer doesn't modify the vehicle and keeps up with the maintenance. They might also want the authority to disable "self driving" on non-compliant vehicles.

> public transit is largely getting there faster than point to point driving

Aside from some niche cases in very dense cities, is this generally true anywhere? I've visited a lot of cities with various levels of public transit, and I can't think of many where it was faster. More convenient sometimes, sure, cheaper, yep, but faster? Not often.

Faster to hands off the wheels, is all I meant.

I was leaning into it being a social thing. If you care about reducing fatalities and such, public transit has been a thing for a long time.

Cheaper: Questionable, unless the customer is unemployed. Tickets are practically always heavily subsidized, the taxpayers pay that bill. Maybe there is a city in the world where this isn't the case but I've never heard of it.

As far as convenience goes self-driving cars should be extremely convenient once we have them, no? I agree with you that public transport makes sense in very densely populated cities but most places in the world are actually wasting money on lots of public infrastructure that makes little rational sense, simply because it's a popular demand.

In the United States at least the subsidies for public transit pale in comparison to subsidies given to drivers in the form of road maintenance and parking subsidies. Subsidized public transit makes a ton of rational sense. The fewer cars there are driving around me the safer I am and the cleaner my air is. I want everyone to have an option to get around that doesn't require a huge upfront investment. It seems you and I are optimizing for different values but there are plenty of rational reasons to support it.

> But, the ones they do have are accidents a human driver would almost certainly have avoided.

> Is this something we would accept?

No, and that's good.

Human drivers behave mostly like humans, even the worst ones. We all have millenia of evolution fine-tuned to recognize human behavior based on the most subtle of cues. So human bad driving is recognizable and thus far more avoidable.

AI drivers are effectively an alies species who make errors that make no sense whatsoever to a human mind, thus they behave, for all practical purposes, apparently completely randomly.

Can this be compared to seat belts? If seat belts needed to prevent injury in every single situation then we would claim that they are ineffective because there is always a chance that seat belt may prevent someone from flying out of the window and not burning in the car being strapped unconscious. Is the goal here to be 100% reliable with 0 deaths or achieve better stats than 46k deaths per year in USA? This now becomes more of a philosophical question.

This is one reason for going to vision only automatic driving systems like Tesla is doing. The system is more likely to fail when a human would also of had a difficult time. Heavy snow, sun blind out, etc. Strange failures due to radar, lidar, and other sensors with not be understood or accepted.

That would be a reasonable argument only if Tesla's image processing were as good as a human brain (it's not) and if their cameras were reasonably comparable to human eyes (they're not). To take the argument to an absurd extreme, you can't cover a car in 240p webcams from 2003 and expect good driving performance.

Moreover, other players also have cars covered in cameras. If anyone thought vision only was the best path forward because they couldn't figure out sensor fusion, they'd already have done it and saved the BOM cost.

For what it's worth, my experience has been that cameras are one of the more problematic sensor systems overall. Vendor software is garbage, any particular tuning is finicky in extreme conditions, you have to clean the damn things, cameras streams take up lots of bandwidth, etc.

> will have 1/10th of the fatal accidents that human drivers would have[0].

If this becomes true then society would have an even bigger problem with organ donations - available supply would plummet. Some of the largest sources of organ donations are from car crash victims.

I always hear people cite this statistic and I’m never sure what they’re getting at with it.

Surely we’re in agreement that this is a good thing, right? That a shortage of organ donors due to the donors not dying is a good thing. Sometimes it feels like people are suggesting otherwise and I can’t fathom the logic.

I think there's a slightly charitable reading of that opinion. Replacement organs are already in short supply, it's a real problem. By "bigger problem" he's referring to bigger than it already is, not that organ recipients dying is a worse problem than all of the organ donors dying like the status quo. It's a good problem to have, but it's still something the medical profession is going to have to grapple with when their supply dries up.

My semi-serious suggestion is self-driving cars should be painted bright orange with big squishy bumpers and a maximum 20 mph speed limit. They would still be perfectly useful as taxis in big cities but it would greatly limit the damage they could do to anyone.

You joke, but this isn't a terrible suggestion. Mercedes has the best answer, in my opinion. Speed limited, carefully controlled environments, full liability resting on the manufacturer. I really don't like how Tesla is foisting off all the beta testing on their customers, pushing that risk to them and other people on (or near) the road, and then also giving them 100% of the liability too.

The incentives are misaligned. Tesla should want to make the software better not just to attract some more customers, but because if they screw it up they're going to be on the hook.

I really don't think so. And we are not nearly close to that situation either. We have some obvious crashes in cars that are nowhere near to be probably "safer then human". And then we have super confident claims of safety by manufacturer.

So far people have been pretty good about accepting self driving cars, even the Teslas on autopilot that crash into parked trucks quite regularly.

> quite regularly

How many times do you think this has happened?

I don't know - I must have seen five or so headlines in the papers but don't really follow the stuff. Here's one https://www.youtube.com/watch?v=LfmAG4dk-rU

Another part of this is that driving is a lot of fun. People (like me) really enjoy it and definitely wouldn’t give it up easily.

Piggybacking on that sentiment, car ownership becomes less an expression of individuality when cars are driving themselves. No point in owning an expensive sports car when it's the one doing the driving. Ride-sharing and fractional ownership start to make more sense than owning the car outright.

> I think the problem we have with self-driving cars is more social than technological at this point.

> it's a hypothetical so give me some leeway on this!

IMO you should not base (and broadcast) your opinions about safety on hypothetical statistics. I don't even believe it's true that overall statistics show self-driving is safer than humans. IIRC prior reports showed that companies were selectively picking statistics about safety.

If 10% of the time they got into accidents humans would have avoided we wouldn't be where we are today. I can't imagine any scenarios where these cars get in accidents that a human would have certainly avoided. You also say avoided accidents don't make the news but I'm pretty sure footage of them avoiding accidents that humans would have no chance of will be a major part of their marketing.

>I can't imagine any scenarios where these cars get in accidents that a human would have certainly avoided.

Then you've not been following the space. The one that immediately comes into mind is the Tesla that slammed into the side of the semi truck because it was painted blue like the sky.

I've been following it pretty closely. I don't consider Teslas to be anywhere near self driving. By saying "these cars" I was referring to Waymo.

> I can't imagine any scenarios where these cars get in accidents that a human would have certainly avoided.

If you follow the events with self-driving accidents, most of them are nonsensical crashes that no human would ever have done.

I follow AV news very closely. I'm talking about Waymo. The small set of accidents they've been accidents in have been limited to fault of other drivers with the vast majority being rear endings

It's not necessarily true for people who think they are in the 99th percentile of best drivers.

As a software engineer myself they will always lose the argument. I saw so many smart systems falter in some weird way that I would never trust a software system completely. I drive an EV with some AI based system that automatically throttles the car etc. But to trust my life and my Familie in the hands of this system (or any other) no thank you.

This is the gist of it, but you have to ask yourself why it's just the accepted wisdom that it's okay to have the massive level of failure we have now.

> will have 1/10th of the fatal accidents

I think I'm relatively safe from cars on the sidewalk. Yet with fsd cars I'm not so sure anymore.

> I think I'm relatively safe from cars on the sidewalk.

Not even true. I've been hit by a car (with human driver) on a sidewalk. Driveways cross sidewalks, and drivers seem particularly inattentive when crossing them. When you're walking on the sidewalk, do you ever walk somewhere that's more than one block away? If so, you're going to have to cross a street anyway.

Reply on news.ycombinator.com