Hacker News Reader

Comment by atty | Hacker News Reader

This mixed messaging of “you don’t need to be looking at the road!”, “but you should be looking at the road.” Is going to lead to more deaths. As someone who works for one of the major global car companies, I’m very concerned for how breathlessly we (as an industry) talk about things like “autopilot” instead of “driver assist”, and a million other flashy marketing terms that make it difficult to just understand what the system is and is not capable of handling.

View on HN

YipYipYip. This is the blue wart in the green sea [0]. The system drives until you should have taken over and died. The situations that are detectable as faulty are not the lethal ones typically. As far as I remember, Volvo denied to implement Level3 years ago [1], which is the only grown-up and responsible answer to this problem.

[0] https://www.sae.org/binaries/content/gallery/cm/articles/pre...

[1] https://auto2xtech.com/volvo-to-skip-level-3-autonomous-mode...

Thanks for linking to good source material.

> When the feature requests, you must drive

Is there some kind of expected delay from the system requesting driver attention to the driver assuming control? Ie something like 3 seconds?

That's the problem, no such system can guarantee it gives you a minimum amount of time. Some situations appear and develop in less than 3 seconds (think small child jumping in front of your car from between parked cars). You might only have 1s to react and if you are not "plugged in" and fully attentive at that instant you missed the window to react.

The more you "disconnect" as a driver, the less likely you are to take over in a split second and this has been proven again and again with humans. A system that works 90% of the time is probably the worst because it's good enough to give you confidence but bad enough that it's a false one. The driver is in the position to buy the advertised "you can disconnect" feature but then only be able to use it under explicit risk of harm to themselves and others. It actually increases the mental load as drivers will try to both "disconnect" (type on the phone, watch a movie) while also "paying attention to the road". Of course both can't be reasonably done at the same time so both experiences are sacrificed.

It's like encouraging people to take a taxi after drinking but then expecting them to take over whenever the driver makes a mistake and holding them responsible and accountable for the outcome. You won't enjoy the drink, or the drive.

These half-way solutions are great for marketing and for people who are all about the hype. But not only are they the bad compromises, they're also fueled by bad and incomplete data, and by marketing departments wanting to sell more. Almost no self driving outfit provides data on how many times did a human make the slightest correction while the self driving was enabled. Or out of all possible driving conditions, what percentage of them were covered by those self driving miles.

My personal philosophy is that cars are either self driving or they're not. Meaning they can either match an average human driver in all conditions they're expected to encounter over the lifetime, or they're just assisting the driver. If it's self driving except when it's not then it's just like a student driver. You wouldn't confuse them with being perfect drivers just because of their perfect safety record, given the supervisor corrects their every mistake.

> Some situations appear and develop in less than 3 seconds (think small child jumping in front of your car from between parked cars). You might only have 1s to react and if you are not "plugged in" and fully attentive at that instant you missed the window to react.

Yeah, humans are weak at this task, which leads to numerous deaths on roads. Can we develop a tech for this? Something, that will watch the road and will assist the driver.

Humans already kill 3000 people every day while driving

Politics is emotional, and emotions do not always agree with Utilitarian ethics.

Self driving doesn’t just need to be safer, people also need to feel that it is.

Sorta. Self-driving cars cannot a injure or kill someone in any situation where a human wouldn’t have.

I feel this too. I would rather have a greater risk of death but be in control of my fate.

Americans drive more than 3 trillion miles per year.

That's exactly the sort of task I'd expect an automated vehicle to perform better on. If they can't beat humans at that they're not ready for the public road.

Of course hand-off could be for liability purposes, like

Car: "I'm sorry Dave, I'm about to plow into a school, this is your problem now"

Dave: [looks up briefly from his game of candy crush before dying in a crash]

The problem is, they can't do this today with the necessary classification quality. Elaine was killed, because someone at Uber wanted these cars to drive so badly. It also is about culture and prediction. If the other participants behave differently to your prediction, it can get hazardous quickly at highway speeds. I can't find a better link [0], but maybe it's good enough to get the point across. The term is "warning dilemma" and means that you only have perfect information at the time of impact. Before, the classification if there will be a problem or not is worse, the earlier you ask. But asking early is necessary for the human to have a chance to react.

[0] https://www.researchgate.net/publication/326568066_Towards_a...

From what I remember, Elaine was mostly killed because she stepped in front of a moving car in the middle of the night.

I’m not convinced that’s the best example, since the AI might have actually done better if it had full control of the car instead of defaulting to ‘please take over now’ mode.

If I remember correctly, the cars built in system for emergency braking in this scenario was disabled. So, any other car of that type would have avoided the accident. However, because the AI apparently knew better, it was disabled and someone died.

AI has the problem that not only does it need to observe the environment and determine what to do, but it has to predict the actions of non-rational actor (humans). We aren’t always predictable. AI isn’t always predictable either. The woman who died may have assumed that a car driven by a person would have stopped.

Right. I could have stated it better, what I meant was that the (safety) driver was not paying (enough) attention to the road and was basically driving Level 3 (the car drives and it will beep whenever I need to take over, so just relax and browse the web). It was impossible to react to the short notice and the warning was _not_ given 10 seconds ago. I personally think the system had sensed her as a succession of standing targets and never predicted any velocity vector. Braking for static radar targets is not happening, because of the clutter you get everywhere you would not drive at all. So the misclassification and misprediction was not detectable for the system (in it's world model, everything looked consistent) and it warned too late.

Press release: "While the self-driving feature of our car had been engaged earlier in the drive, the driver was in control at the time of the crash."

>That's exactly the sort of task I'd expect an automated vehicle to perform better on

If the AI could talk, it wouldn't be so much "I'm sorry Dave" as "I cannae change the laws of physics!"

An AI can think about the situation for several million instructions worth, if it is running at >GHz speeds, but it can't prevent a crash given a few milliseconds, because inertia. And for the same reason, it's futile to hand off to a human.

This sort of assistance is an excellent use of current technology, and a way to move forward to true autonomy. What needs to stop here is the two-faced, fatally ambiguous marketing, and if manufacturers cannot act responsibly, regulation will be needed (more than the current regulation for responsible driving, which only kicks in after the damage is done.)

Who would the car optimize for? Saving the kid or saving the driver from getting rear-ended or swerving into an object?

See kid => hit brakes. This is already way better than a driver who doesn't notice the kid and doesn't hit the brakes. The inside of the car is very well designed to keep the occupant alive in case of collision, the car should prioritise not hitting other people. I hope self-driving technology will at some point be regulated regarding what the car should attempt to do in those situations. With more vehicles having collision avoidance technology, the car behind you won't rear-end you because its AI will react fast enough while also refusing to drive dangerously close to the car in front.

Ok now bonus Problem. The car has to swerve and either hit a kid or an old lady, which do you hit and why?

I wouldn't have asked the question if it was that simple :) If you have no other cars around you, are traveling less than 25mph and have plenty of stopping distance, sure, 'see kid => hit brakes' should work to save everyone in that circumstance.

If your assumption is that you won't get rear-ended because everyone will be using automated collision avoidance technology... Well, we're at least 15 years away from that.

As someone who bikes and is used to seeing every other person on their phone while driving, fully self-driving vehicles can't come soon enough. But we need to have real discussions about who the self-driving vehicles will save in regular circumstances. I'm skeptical about them optimizing for, or even having the ability to optimize for saving people outside of the vehicle over protecting the occupants.

The systems probably already have some kind of an estimate of "surprise possibility in the next 3 s".

For example on a narrow road with a building right in the corner, going around the corner, anything might happen. You don't know what's around the corner. So the system will either go so slow that there's ample warning time or then require driver attention already before attempting to turn, because the driver needs to be ready to react to potential up-and-coming new information fast.

In contrast on a wide highway, it's a more easy-to-estimate environment, visibility is better etc. So as long as other cars are far away and the velocity differences are not huge etc, the driver can stay inattentive.

The challenge self driving cars face is that being "overall better" is probably not enough. Regressions in certain segments would hardly be acceptable for anyone. You can't tell people "FSD cars lowered overall deaths by 10% but we now kill 30% more toddlers because [technical reason]", or that "highway deaths are going up", or basically any segment that anyone could reasonably care about. Cars have to match and exceed human drivers in every category for them to be acceptable, and even 99% there may not be enough.

Another point is that there is a psychological difference between sitting behind the wheel in a vehicle knowing that around 10 in 100,000 people get themselves killed each year, and then sitting in a machine that kills 10 out of 100,000 people each year.

In the first situation, you feel in control, and at least think that you can do what is necessary to avoid that you end up in the statistics.

Personally I would be very skeptical about trusting my life to a machine that has a non-zero chance of getting me killed, even if the machine, on average, performs better than a human, because most of us think that we are better than the average driver.

Unless, you know, there’s a reflective truck passing by. Or faulty recognition of lines on the road steer you straight into a barrier.

>situations appear and develop in less than 3 seconds (think small child jumping in front of your car from between parked cars)

I trust full self driving tech to handle this situation better than a human driver more often than not. The computer doesn't get distracted and has better reaction time.

The Tiger Woods driving experience as informal industry use case standard to focus on is as good as the informal two golf bags carrying capacity.

> full self driving tech

Full self driving tech is presumably (and by definition) fully able to handle things by itself. We're talking about "partial self driving tech" (SAE L3) which relies on the driver taking over when it doesn't know what to do. The context switching for a regular human absolutely kills the reaction time in these situations making this a "I'll mostly drive myself but when I can't you're almost guaranteed to cause a crash" type issue.

>YipYipYip...

Is this a Sesame Street reference?

https://youtu.be/KTc3PsW5ghQ?t=97

YipYipYip =¦-D

Com...pjutaaa.

Volvo is absolutely making the right call here. There is too much irrational exuberance around SDC.

The obvious answer is to apply AI to the entire system, not just the car navigating the road

* Is the driver paying attention, how much attention should they be paying verses the conditions?

* What is the environment like, how should I react, refuse to drive? Go slowly? Honk?

We already have self driving vehicles, they are called Mules. Why not make artificial mules instead of artificial pole position. The car should be an active participant, more of a copilot and than a blind automaton.

Self driving cars should be shown to have passed a repeatable rigorous adversarial gauntlet before being allowed on the road. And the individual cars should have to be re-certified every 6 months by retaking the test.

As a side note - I think humans should require retaking their test on a regular basis. Maybe every 5 years, and just a short 'refresher' test on the way home from work. It would help find those who simply can't drive safely anymore, and may dissuade others who don't think it's worth going through that process. It would also take a lot of stress and anxiety from the problem of families who know mildly-demented grandad really shouldn't be on the road because he can hardly see at night, but find it hard/impossible to actually do something about it.

there's a deeper problem here in the US. a large portion of the workforce needs to drive to work. a lot of these people are not very safe drivers, through some combination of incapability, unwillingness, or ignorance. it's hard to strike a good balance between keeping unsafe drivers off the road and not upending people's livelihoods.

Also the driving test is a joke in the first place so there are a ton of unsafe drivers.

I'm sure this varies state-to-state. I wouldn't say my driving test was a joke (in the sense of easy), but the hard parts had very little to do with driving safely. the whole thing took place in a parking lot at 5 mph. I failed the test my first two times because it was really difficult to perfectly parallel park my dad's massive truck.

I think it would be okay to make the initial test a little harder, and ideally more focused on real world driving safety. the real problem is if you let someone have a license, they plan their life around it, and then you yank the rug out from under them because they couldn't back up 100 feet in a straight line ten years later.

In my state, the test never touches the highway or any busy roads so you don't have to learn how to merge, stay in your lane, drive at speed, etc. Actually most highways around explicitly prohibit permit holders. I personally know more than one person who had their first experience on the highway after they had a license and it was an absolute disaster. They were white knuckled, driving 20-30mph under the limit and every merge was a close call. I think the only thing they actually test is whether you can parallel park or not.

In Indiana we went on the highway in drivers Ed, which had a speed limit of 55. Then we crossed for a day into Michigan because their state limit was 65. Indiana has since raised the state limit...

Maybe getting to work should be an incentive for practicing safe driving. You cant get to work if you crash your car either.

sure, that's not unreasonable. but with the current situation in the US, you would need to drastically expand welfare, public transit, or both if you really want to lean into the "driving is a privilege" idea.

Yeah I don’t think “drive safe or lose your job” makes for a good billboard.

This is the blue wart in the green sea.

The system gives you more power and you'll kill yourself. Similar situations with horse-drawn are not as lethal. As far as I remember, Volvo denied to use horseless carriages years ago, which is the only grown-up and responsible answer to this problem.

I do not see how this gives more power to the driver. Decreasing responsibility is not equivalent.

Edit: sp

I blame this largely on Telsa essentially driving the industry like a heard of cattle. This is/was a great when it came to accelerating the development of electric cars, and I realise that the software development community really likes the "move fast and break stuff" mantra, but there are reasons why engineering fields with chartered engineers have processes like FMEA: people die if things fail.

A friend who is an engineer at a premium car manufacturer working on sensors for self-driving is telling me that the internal policy inside the manufacturer is, that the only competitor that matters is Tesla, so they only compare themselves to Tesla. The same friend also believes that real self-driving is still many years out, even sensors that can deal with general weather conditions do not exist yet.

How did your friend arrive at the conclusion that we don’t have good enough sensors?

IMHO AI to analyse the data is what’s missing - we have cameras that are just as good as human eyes, so we know for a fact that the current sensors are enough to drive in general weather conditions.

Cameras don't give you depth.

You have two choices:

o have a sensor that gives you depth directly (ie laser, radar, or lenticular array) o try and infer depth from stereo cameras

Lidars are expensive and power hungry, Radar is cheap and mature, but doesn't give you anywhere near as much resoultion

lenitucular only has a tiny range. There are other time of flight sensors, but they are either not production ready, expensive or both.

What Tesla have chosen to do is kinda try and merge radar and monocular object detection to give a higher frame rate depth estimation of objects. However its expensive to develop, unreliable, and terrible in corner cases (ie, if sees an unknown object, it can't place its depth.) Now humans can do this, because we've had years of training, Tesla can't.

Tesla's fancy cruise control is dangerous. Its auto drive stuff its trying to develop is even more dangerous. Instead of finding the sensors and processing spec needed to drive safely, they are trying to use the sensors and GPU they already have. Its not going to work and is unsafe.

> Now humans can do this, because we've had years of training...

Humans can mostly do this. Like 90% of UFO sightings are people seeing the moon and getting confused and thinking it is a nearby object following them.

There are surprisingly sophisticated (and precise) machine learning algorithms for mono-camera depth estimation. It's not a problem anymore. Furthermore, most companies use various focal length lenses. The real problem lies in sensor fusion. Which sensor's input do you believe most, especially if one or more is occluded/dirty.

I have yet to see any ML algorithm that can reach 99.999% accuracy or better (we need better).

Why do we even need such accuracy? Humans are extremely bad at estimating distance and nevertheless are quite ok drivers. ML needs to make sure that the car is not bumping at things or people, which is a very different task.

Yes, so "can/cannot bump into" is a classification tasks which needs quite some accuracy.

A human on the other hand can recognize another human with near perfect accuracy.

Because the failure modes are different. Humans with basically 100% accuracy both correctly recognize depth and put it into the right bucket of near/far but are woefully bad at estimating the numeric distance. So you’re right that you don’t need to do better than extremely coarse estimates like close, a few car lengths, in the distance, etc..

If computer vision fails at all to recognize depth (semi truck with clouds painted on it isn’t recognized) or misclassifies whether something is near or far that’s much more dangerous.

> There are surprisingly sophisticated (and precise) machine learning algorithms for mono-camera depth estimation

There are, and I wouldn't trust them with life critical stuff. The more accurate ones are not realtime. The hardpart is that they are very noisy. unknown objects wobble about in depth considerably. You need to do lots of filtering to get useful results, which eats into time.

It is very much an unsolved problem. Its one that I'm partly working on now. However for the device that I"m partly working with, monocular estimation is far too power hungry, noisy and generally shite.

> Furthermore, most companies use various focal length lenses.

no, tesla _rely_ on having a wide, medium and zoom camera. However they blended into the same effective sensor to give a better chance at tracking objects. I bet you a $10 that if you block one, the whole system turns to shit.

> The real problem lies in sensor fusion.

Sensor fusion is trivial. Its understanding what they sensors are telling you, thats the hard part.

for example, when trying to turn across oncoming traffic, all the sensors will tell you that stuff is coming towards you, what they won't tell you is if its safe to cross. Thats the hard part. Given that tesla can't accurately place a car on the road yet, they can't safely cross traffic.

Sensor understanding is the problem that is really unsolved. Depth estimation with stereo cameras is 85% of the way there, monocular estimation is no where near that level, simply because robust object recognition isn't going to be a thing for at least another 6 years.

A camera is a light sensor, a camera is not an object sensor. Cameras as light sensors definitely exist.

A camera feeding into phenomenally sophisticated AI trained on all sorts of weather conditions with windshield wiper smear and glare and everything else, which can turn those horrible images into a picture of what actually lies ahead, is an object sensor. Those aren't good enough yet.

Goodness gracious doesn't anyone in the auto industry ever even look at how the word "autopilot" has been used in the aviation and marine industries? It has _never_ meant that the vehicle operator gets to abandon their responsibilities.

However I agree that the mixed messaging is sad and irresponsible, and this article's author could have been more circumspect.

How the term "autopilot" is used by professionals in the aviation and marine industries isn't the main point here. What matters is what non-professionals think when they hear the term "autopilot". If you market something as "autopilot" don't be surprised that general consumers don't have the same nuanced understanding of the term as professional pilots.

Right, and this is what Tesla autopilot is. Tesla no attention dream they want to someday roll out is under FSD (full self driving).

You're right, but pilots are highly trained and must maintain their training/skills. On the other hand, it's possible to get a driver's license without any formal training (in many Western countries at least) and then buy a "self driving" car without any training on how to use it.

Because of that divide, I think there should at least be an online training curriculum and test before your car will unlock autopilot mode.

> talk about things like “autopilot” instead of “driver assist”

I actually find driver assist technologies almost as damaging. For example, lane assist (car centers itself in lane) can cause the car to veer when lanes are mis-painted. After having it turned on for a few minutes without issues, you begin to relax, then the car decides to take an off-ramp or swerve towards the shoulder.

As a comparative example, let's look at automatic gear boxes vs manual. As far as I know, there are no auto gear boxes that will occasionally require you to hit the clutch in order to successfully change gears. Their either completely auto, completely manual, or auto with manual override. Having something halfway between automatic and manual is just ASKING for problems; that being said, an auto gearbox that is expected to work only 99% of the time also really doesn't exist, which is why self driving is hard.

I'd much rather better alerting and imminent danger capabilities; more ubiquitous and accurate blind spot sensors, lane change cameras, brake alarms (or even auto-braking, if it's high reliability). These technologies will allow the overall self-driving landscape to improve (because there's so much overlap; self-driving NEEDS all of these sensing techs, and needs them to be REALLY good in order to work) and mature over time, while road infrastructure, mapping, legislature, and public opinion catch up.

Humans mess up mispainted lane lines also. Have seen many times where there were 2-3 conflicting lane lines painted and people just kinda drove where ever. I think both humans and robots could benefit for federal laws about road construction and lane markings.

> Humans mess up mispainted lane lines also. Have seen many times where there were 2-3 conflicting lane lines painted and people just kinda drove where ever.

That's very true, I've seen it as well. I think the big difference is that humans have the context and experience to deal with those situations on the fly, making choices that a self driving systems just haven't matured enough to make; this is why I advocate for better alerting systems rather than anything that changes the trajectory of the car on its own.

I’m sure they will continue to sell alerting systems for many years. I hope I don’t have to use them. I quite enjoy autopilot on long trips.

Yes, it should be quite simple:

Self-driving: car comes with no steering wheel.

Lane/drive assist: car has steering wheel and you have to steer the whole time—with occasional nudges from the AI.

>Is going to lead to more deaths.

It'll lead to the same number of deaths and slightly more complicated lawsuits.

At a statistical level Nobody(TM) is heeding fine print warnings.

The Honda is level 3 only during traffic jams -- it's limited to 50km/h and only on the expressway. If it's completely incompetent the worst that will happen is some minor body damage. It's not going to cause any deaths.

> how breathlessly we (as an industry) talk about things like “autopilot” instead of “driver assist”

Is this true? I was just purchasing a car and did a lot of research. I was looking only at the European brands because I like the overall offering. There are different names, but none too flashy or promising too much.

How they call the driving assistant systems:

- BMW: Driving Assist (Regular/Plus/Professional)

- Volvo: Pilot assist

- Mercedes-Benz: they just describe the functions, i.e. active cruise control, steering assist, etc.

- VW group (VW, Skoda, Seat): Travel assist

It's because they know that Self Driving Cars are the way to Guarantee a revenue stream for the life of the vehicle.

Screw actual utility, this is about revenue, plain and simple.

From my observations, it seems the most boastful (Uber, Tesla) are behaving the most recklessly. Waymo's driver is probably 10x better (however you want to measure that), and they seem the most measured and careful in their communication.

More important than short run death is longer term road deaths. Their messaging is flawed but hundreds of thousands die in car accidents yearly... The rollout of these cars needs to be smooth and quick to prevent longer term death. It's incumbent upon early adopters to be responsible for the sake of society.

Regarding this, I strongly believe the correct approach is L3 capability but only expressed through L2 features.

We're not at a point where it's safe for drivers to move their attention from driving, but the technology is mature enough that it can and should intervene wherever possible to avoid collisions and other incidents.

The expectation that drivers can divert their attention from driving to perform other tasks on the expectation that they can resume control at short notice is extremely misguided. This should not prevent similar sensor and software tech that could enable L3 autonomy from enabling more effective accident avoidance technology that intervenes in situations where the driver may drive in a careless or dangerous way.

Meh. It only needs to be better than the average driver. Several companies have already achieved that

I think the bar needs to be much higher than "average".

Although beating the average is arguably "good enough" to deploy in a collective sense, we live in a society of individual actors, and if I'm an above average driver, it needs to be better than me, not better than average, for me to want to use it.

Assuming a society in which each person individually and rationally chooses whether or not to use it, if you want 99% of people to use it, your software needs to be better than the 99th percentile driver.

This assumes a lot about the correctness of people's perceptions of their own driving skills. My experience is 99% of drivers think they're above average, and that obviously can't be true.

In my experience there's two kinds of people who'll say their good drivers and they're both good but with wildly different definitions of "good"

You've got Jose the 35yo MRI service technician who logs 100k/yr for work and has had so much time to get good he can tell exactly what traffic is doing and are highly capable of printing what is about to happen and are preemptively making moves based on that. He knows his insurance company would crucify him if the saw how he flings an overloaded Transit Connect through an on-ramp or parallel parks by braille so if you press the issue he'll tell you he's good at getting where he's going but that the bureaucrats who write the state driver's manual wouldn't like him.

And then you've got Karen the elementary school teacher nearing retirement who logs 10k/yr 5k of which are spent looking at her phone. She has spent a cumulative one hour of her life above the speed limit despite spending much more than that on highways where the traffic flow is well above the speed limit. She follows every rule in the book to the letter, gets honked at daily and once a week she has a story about "some asshole" she got in a conflicting situation with. She doesn't know how far down the gas pedal on her 4Runner goes but oh boy does that brake pedal get a workout when "oh crap almost missed my turn". She swears up and down that she's a good driver because of the seventeen fender benders she's been in she was only at fault in the three that were caught on camera or where witnesses stopped.

Which definition of "good" do you want near you when there's 3" of snow on the road?

> Which definition of "good" do you want near you when there's 3" of snow on the road?

When there is snow on the road you should not be near either of them.

You should be far enough behind that you can stop when the car in front gets into difficulties. The rule of thumb on a dry road is that you should be three seconds behind. At 100 km/h (about 60 mph) that's 83 m (about 270 ft), say 17 car lengths (for my Tesla S that is). If there is snow on the road perhaps it would be wise to allow more distance.

See, for instance, https://www.driveincontrol.org/drivingtips/the-three-second-...

> My experience is 99% of drivers think they're above average, and that obviously can't be true.

It actually can be true, depending on the distribution of skills you assume. Say you have 100 drivers, 99 with the same skill level, and one driver which is worse. 99% will be better than the average. I'll see myself out ;)

If you really think you're better than a self driving car, then don't buy one. Or do you want the gov't to choose for you?

You should be happy that other people can buy them because they'll no longer crash into you ;)

The average driver got to that level after years of practice. With this system in place, the new-average-driver will have the skills of someone who's just barely gotten their drivers license years ago and have never used it since. And that person will be expected to pilot the vehicle only in times of such dire conditions that the computer cannot do it.

Perhaps the use of this system should be licensed separately from a standard drivers license, akin to an IFR rating. And to keep the rating, manual driving must be performed periodically and logged as well.

If we have evidence that this is an issue, I'm on board with that.

If it turns out this is not an issue and we unnecessarily made self driving cars less common, that would have the same effect as shooting a few hundred random people every day (3,700 people die every day from crashes)

We'll find out in a couple of years

> 3,700 people die every day from crashes

Can you provide your source please? Closest I could find is https://www.cdc.gov/injury/features/global-road-safety/index... which states that “[e]very day, almost 3,700 people are killed globally in crashes involving cars, buses, motorcycles, bicycles, trucks, or pedestrians. More than half of those killed are pedestrians, motorcyclists, or cyclists.” I didn’t find a number for just motor vehicle crashes, and since the rates are three times higher in developing countries, it seems like the cost of ownership is going to be a huge factor.

There is also the case of "liability". An average driver can drive the car into a tree with no problems at all. No one except the driver is responsible there.

But if a company sells a product with the name "Autonomous Driver", than that company is liable if the product mulfunctions (ie. drives a car into a tree).

Let the courts decide who's liable. I'm sure that in practice this would be decided on a case by case basis with logs from the car's computer.

But if the car is in full auto mode the driver is not expected to pay attention or look at the road. Based on the situation the car thinks it can drive autonomously.

At least a partial liability will have to fall on the OEM that's why nobody is making any binding promises (including Tesla)

would be decided on a case by case basis with logs from the car's computer.

If I was a car manufacturer and I knew that my log files would undoubtedly be used as evidence against me in a criminal negligence trial, I would think very hard about what I did and didn't put into those logs.

Yes in California/Arizona sunny weather on wide US roads with. My experiences with even driving assist on small twisty road in the snow tells me me things are nowhere close. The car was accelerating in situations (e.g. coming over the crest of a small incline with a corner at the end), where no driver would ever accelerate and which would have resulted in some bad situation without intervention. We have seen Teslas getting confused by fork in the roads on the highway.

I don't think anybody's sanctioning the use of driver assist features on small twisty roads the snow yet, are they? If so please share what you were driving - that would be interesting even if it didn't work well at the time. Most of these things, AFAIK, are in the LKAS, ACC, LDW range and meant for highways and there are mutterings about that in the manual. I sometimes use ACC and/or LKAS on country roads but only with very low expectations.

Teslas at least get better over time. If it got confused over a fork in the road, there's a decent chance that after a near-future firmware update that same car will no longer be confused. A few folks have acknowledged that they're now smart enough to slow down for curves (even sometimes a bit generously), and they are among the few that work pretty well off-highway (most of the time).

It may not being marketed directly, but calling something "full self driving" greatly implies that self driving is fully supported in all scenarios. I have not seen any Tesla marketing that says "full self driving, on big roads with clear markings in dry/slightly damp weather".

Words matter

Actually, it only needs to be better than the average mammal.

And I am not sure any level of AI has reached that level yet.

It doesn't lead to more deaths. See Tesla for proof. This is a super bad take.

https://www.tesladeaths.com/

Have you mentioned how sparse the autopilot claimed column is? And how even sparser the verified autopilot is? Seems like the authors are trying to inflate the total numbers very hard.

Not to mention the spreadsheet admits Autopilot was released in 2015 but they are still including deaths from 2013.

This can't be more blatant.

Autopilot might not be around, but up until very recently Tesla refused to participate in a lot of auto safety testing saying that they believed the testing regime was "flawed".

I have no issue with keeping track of auto deaths from a company who is claiming that their vehicles are safe while preventing anything but the legal minimum necessary tests from occurring.

Ok, reduce that to where autopilot is engaged.

It should be noted that that site is run by a group of pseudonymous people who do not hide the fact that they have been shorting Tesla for years. They count every death involving a Tesla. If someone jumps in front of a manually piloted Tesla and is run over before the driver can react, it is added to the spreadsheet. They’re holding the company to an impossible standard. Moreover, they don't actually care about reducing deaths involving Tesla vehicles. They want Tesla to fail so that they can profit. It’s a purely selfish act.

The most prominent member of the TSLAQ people and (IIRC) the first to publicize a spreadsheet of Tesla deaths is Elon Bachman (a pseudonym). He has also been a charlatan on COVID. A year ago he said, “After 4 months of white hot Coronavirus panic, the disaster is visible everywhere except in the data. Total deaths, rounded to the nearest percent, remain 0% of annual flu deaths, and serious cases are falling”[1]

I tried to bet him up to $1,000 that US COVID deaths would exceed 25,000 by the end of 2020.[2] He ignored me and continued to deny the harm caused by the disease.

One simply cannot trust anything he’s involved in.

1. https://twitter.com/ElonBachman/status/1237030292340314112

2. https://twitter.com/ggreer/status/1237164317142736896

> They want Tesla to fail so that they can profit.

A more realistic take is: They think Tesla is massively overvalued and should be held accountable for fraudulent corporate behaviour.

> He has also been a charlatan on COVID.

Kind of ironic given that Elon himself was (is?) one of the first vocal Covid deniers.

Check out TC's Chartcast [1] in case you are interested in TSLAQ, beware it's a deep rabbit hole.

I've previously tried to sum up Tesla's red flags here [2].

[1] https://www.buzzsprout.com/758369

[2] https://news.ycombinator.com/item?id=26065075

> They think Tesla is massively overvalued

> fraudulent corporate behaviour.

These two things have nothing to do with each other.

> and should be held accountable for

Holding a company accountable should go through the courts, not personal profit.

I'd be happy if short selling weren't an option. If you believe something is overvalued, stay out. You're not wanted.

If you believe a company did something wrong, file a lawsuit.

You haven't made an argument against short selling.

Pessimism is counterproductive to the long-term development of humanity and rapid deployment of electric vehicles and clean energy. Armchair pessimists don't deserve a place in the stock market. If you think Tesla is doing something wrong, either (a) go work there and fix it or (b) start your own Tesla competitor. Short selling isn't constructive.

What's wrong with short selling? The only reason why people like Elon Musk don't like short selling is because they know that their companies are massively overvalued.

The problem is profiting from spreading misinformation. Tesla short-sellers run large operations that fabricate and spread lies. It's usually easier to spread misinformation than to debunk it.

And the companies don't spread misinformation? Not saying it's right in either case but companies, Tesla included aren't shining beacons of morality and truthfulness.

It's not in their interest to lie in the long term. Those companies who lie are scams and they're spotted pretty quickly (e.g. Theranos, Nikola). Elon has made some mishaps, such as 'funding secured @420' tweet, which he was punished for. Public companies have to be pretty careful in their communications. Making promises with too optimistic schedules is not lying, it's just an error in forecasting, and is common in technology.

That couldn't be further from the truth. Theranos survived for 15 years. Wirecard 22 years. Enron 10+ years. Fraud gives you a massive competitive advantage.

Not too along ago I thought of Musk as a misunderstood genius, now I'm pretty certain he knows exactly what he is doing (for the most part). If you look at all the oddities surrounding Tesla, there are clear patterns emerging.

Plainsite has a good summary [1].

[1] https://www.plainsite.org/realitycheck/tsla.pdf

That report is a bunch of horseshit. Lot of words without any substance. He even cherry-picked some data to "prove" that Tesla's sales are declining. Everyone can see the actual progress that Tesla and SpaceX has made. Their cars are winning awards and they're innovating and building new factories as fast as they can. Who cares if they miss a couple of estimates. SpaceX can deliver payload to orbit with much lower cost than competitors.

True though that those companies survived for too long.

I don't know what world you've experienced, but lying in business is incredibly common. I've seen it personally, and there are plenty of cases of Tobacco companies, DOW, Alcohol companies, and many many more knowingly lying.

Of course they do, it's in their best interest to lie. To think that companies will only tell the truth "because it's in their best interest" is childish, and not at all historically accurate.

And Tesla will happily release misinformation too.

One of the last fatalities, Tesla was more than happy to push out a press release based on telemetry, saying "Autopilot wasn't at fault, the driver was inattentive - the vehicle even told him to put his hands on the steering wheel!".

They somehow neglected to mention that the steering wheel alert was triggered, ONCE, and FOURTEEN MINUTES before the crash.

Misinformation is not a good thing. But lets not pretend that Tesla is some downtrodden underdog just trying to make our lives better.

Also, if you have an accident in your Tesla, you'll have a lot of fun trying to get any telemetry information from them, even if Tesla isn't a named party and you're just dealing with the other involved person. You'll need multiple subpoenas and expect them to resist releasing any data as "proprietary".

But should your telemetry from an accident be able to be spun (correctly or otherwise) into a Get out of Jail Free card for Tesla, expect it to be released to the media without your consent or authorization (I'm sure it's buried in Section 48, Subsection 24, Paragraph 14c iii that you consent, but still).

Musk has never been a covid denier. He understood that covid wasn't as dangerous as many claimed, especially for younger and healthier people. The only thing close to covid denialism that I can find is Musk's decision to resume production at Tesla's Fremont plant in defiance of Alameda county lockdown orders. This was in May of 2020 after months of failed negotiations with government officials. It's important to note that unlike Bachman, Musk had skin in the game. He asked that if the government sent cops to shut down the factory, only he be arrested.[1] The factory resumed production and nobody was arrested. A week or so later, Alameda county changed its rules to allow the factory to operate legally.

1. https://twitter.com/elonmusk/status/1259945593805221891 "Tesla is restarting production today against Alameda County rules. I will be on the line with everyone else. If anyone is arrested, I ask that it only be me."

Here's a selection of tweets for you, by a celebrity many still trust and consider a "scientific genius":

@elonmusk Mar 6, 2020

The coronavirus panic is dumb

https://twitter.com/elonmusk/status/1236029449042198528

@elonmusk Mar 19, 2020

Based on current trends, probably close to zero new cases in US too by end of April

https://twitter.com/elonmusk/status/1240754657263144960

@elonmusk Mar 19, 2020

Kids are essentially immune, but elderly with existing conditions are vulnerable. Family gatherings with close contact between kids & grandparents probably most risky.

https://twitter.com/elonmusk/status/1240758710646878208

@elonmusk Jun 29, 2020

There are a ridiculous number of false positive C19 tests, in some cases ~50%. False positives scale linearly with # of tests. This is a big part of why C19 positive tests are going up while hospitalizations & mortality are declining. Anyone who tests positive should retest.

https://twitter.com/elonmusk/status/1277507826529660928

The only screw-up I see is his prediction that the US would conquer the spread by the end of April. All the other info is pretty accurate, especially considering how little we knew back in March. Compared to most authorities and experts, he did a pretty good job. Remember two weeks to flatten the curve? Remember when the official line was that masks don't work?[1][2][3] Remember when the WHO said that travel bans are a bad idea?[4]

Musk was calling the panic dumb at the same time as the WHO was saying, "Our greatest enemy right now is not the coronavirus itself. It’s fear, rumours and stigma."[5] If we judge him by the same standards as every major institution (media, governments, NGOs), Musk is far less deserving of criticism. And unlike those institutions, Musk never claimed to be an expert on the topic.

1. https://twitter.com/WHOWPRO/status/1243171683067777024

2. https://twitter.com/UNGeneva/status/1244661916535930886

3. https://web.archive.org/web/20200312104152if_/https://twitte...

4. https://twitter.com/WHO/status/1224734993966096387

5. https://twitter.com/WHO/status/1233418231261646849

That's an interesting table. I would say that it reinforces the case that Autopilot (actually labelled Autosteer in the UI of my 2015 Model S) is actually not dangerous.

I presume that the aim of the site is the opposite though.

The stats I’ve seen quoted about Tesla are super misleading and not at all apples to apples comparisons.

The issue isn’t whether having the technology saves more lives than not having the technology. The issue is whether, given the technology, the marketing creates a perception that results in unsafe usage of the technology, costing lives.

It’s is easy, even almost trivial to drive long hours on almost nothing happens freeways. We don’t have any sort of statistics on the dangerous close encounters, and on whether these remain close encounters with teslas.

We have all sorts of statistics, every car maker keeps track of disengagements and events. What are you talking about?

Accidents per miles is not too informative. For example do teslas fair better than an average human driver on eg. someone running a red in front of the car? On a suddenly overturned car on the freeway? Do we have enough data to answer these questions?

I don’t understand this kind of question. Obviously you need some statistical measure to compare safety, and incidents per mile (or vehicle years, or hours driven) is an accurate way to estimate accident rates. All we care about is that on average it crashes less than human drivers. If there are specific conditions where it fares worse those would be pretty obvious to address, and naturally the odds of when crashing the situation being weirder than usual is a given if safety improves for the average case.

That aside, the answer to the first one is yes. Because of sonar and the cameras, a Tesla can see traffic two or three cars ahead and will initiate braking way earlier than a human would in the case of someone else running a red light - there a few videos of this exact situation available in YouTube. As for the second, probably not enough data, but in absolute numbers humans are responsible for nearly al of those cases so far, they happen weekly.

By the way, the overturned truck was not a fatal accident - the car triggered emergency braking and the driver came out without a scratch. The fatal one was a couple years ago when a Model S ran under a white truck making an unsafe u-turn.

Incidents per miles is fine with a big enough data set —- but I disagree that it is enough in case of a new technology that can potentially kill. Like let’s say teslas are absolutely safe on the highways, much more than human drivers, but would have a tendency to hit pedestrians much more so than humans. It is possible that it will have a better incident per mile record even though no sane person would legalize them in this case.

> If there are specific conditions where it fares worse those would be pretty obvious to address

Like, if (inSpecificSituation) { payMoreAttention(); } ? This is the actually hard part of the problem, not breaking when something is close and stay in the lane. They can’t even create adequate test environments for the many many special cases that can trivially happen in a city.

Also, how does it see traffic two or three cars ahead? It sounds like a marketing gimmick but correct me if I’m wrong.

> Incidents per miles is fine with a big enough data set

Only if you're comparing comparable conditions. If you compare autopilot in the sun to humans in the snow, infinite miles won't make it any more valid of a autopilot/human comparison.

Tesla's miles driven is self-selecting, because AP won't engage when it can't because of poor conditions.

Humans don't get that luxury.

You can't say for Tesla (deaths/miles driven on good-for-AP roads in acceptable conditions) and for humans (deaths/miles driven on all roads in all conditions) and then act as if they are comparable or prove "improved safety", because they don't.

> All we care about is that on average it crashes less than human drivers.

All we care about is that on average it crashes less than human drivers in comparable conditions. If you compare a 40 year old driving a tesla on autopilot on a sunny freeway to an 18 year old driving a 1995 beater without modern safety features, no ADAS, etc, in a snowy busy intersection, autopilot doesn't have to be that great to look better. If you compare tesla on autopilot to "average human miles" you rolling some very invalid comparisons into the averages. You can only usefully compare averages if they're averaged over comparable contexts.

A less extreme comparison, if my memory serves, is tesla rolling out ADAS with autopilot when they did a before/after comparison. They compared autopilot-available + ADAS-engaged to autopilot-unavailable + ADAS-unavailable, or something along those lines, if I remember correctly.

The issue is when there is a crash or a death could that situation have been avoided if the driver was looking ahead with two hands on the wheel, like you would in any other car without the marketing hype.

I've seen more then enough crashes and deaths in Telsas that were perfectly avoidable providing the driver wasn't watching their ipad.

Really? I've been driving for over forty years (about 15k km per year) and I have never seen a death involving any car let alone the relatively rare Tesla.

I've seen emergency vehicles attending a crash perhaps once or twice a year but never seen the crash occur. In fact the only ones I have personal first hand experience of is the time I rear ended a car, the two times that I was rear ended, and the time I slid off the road on an icy corner. All three rear ending events were relatively low speed incidents (under 30 km/h) and would quite likely have been mitigated or even completely avoided if the cars in question had had automatic emergency braking. The icy corner was my own fault for not thinking ahead.

So, to me, your statement, without some more context, sounds like an exaggeration.

I too have been driving some some time now but we have this thing called the internet, where you can see and share information.

Lemme help you out friend - https://www.youtube.com/results?search_query=tesla+autopilot...

So you didn't mean: "I've seen more then enough crashes and deaths in Telsas"

you meant: "I have heard about .." or "I have seen accounts of .."

Those are not quite the same thing.

I'm not sure why you are going downvoted, this is true.

Tesla's miles have the advantage of being inherently autopilot capable, and turning off when things get dicey for it.

Human drivers do not have those advantage.

So Tesla is comparing drivers in situations where AP doesn't even try to drive the vehicle because it couldn't.

This is amazingly misleading. And Tesla knows it.

It doesn't say you should be looking at the road, simply that you are aware of your surroundings and that you remain ready to retake control should the vehicle notify you should.

That's different.

How can you keep aware of your surroundings going 60+ km/h without paying attention to the road? Even more if you are required to take control in an instant, you have to be completely enveloped by your spatial awareness.

I have raced go-karts, I've raced GT cars, there is absolutely no way to keep aware of your surroundings if your focus isn't constantly on the road and checking mirrors, full stop.

The language here is critical. The question is where your focus lies, and at level 3 autonomous driving, there are situations where your focus can be elsewhere besides the specific road conditions.

You are assuming “aware” means “ready to take over in an instant”. That is not what it means in this context.

> talk about things like “autopilot” instead of “driver assist”

As stated countless times before, autopilot in planes isn't even geared towards handing most flight scenarios or challenging conditions. Tesla is technically correct to call it this, even though this naming is confusing to consumers who think "plane flies itself" and think it means their car can drive itself in all conditions and avoid at-fault incidents - which most likely will indeed lead to more deaths.

The consumers are not techsavy like Tesla or HN world. Every word used for marketing should be wisely thought or we will end up with consequences

What are the rules to this "technically correct"?

It's not even flying the car!

You need the latest Tesla firmware upgrade to activate flight. Or an unfinished bridge combined with lane tracking and an inattentive driver.

I understood that reference.

https://jalopnik.com/a-tesla-stan-dmd-me-to-show-what-they-t...

> As stated countless times before, autopilot in planes isn't even geared towards handing most flight scenarios

As stated countless times in response, it doesn't matter what autopilot actually does when the concern is the public perception of the marketing.

If "autopilot" sounds to the general public like "it drives itself" (which is the simple etymology of the word), then it doesn't matter one flying fig what autopilot on planes actually does.

Why isn’t there the same level of hand wringing over Ford’s “co-pilot”? Surely a copilot is even more capable than an autopilot.

This assumes that no one is hand wringing over Ford's marketing. I haven't seen as much of Ford's marketing as I have as Tesla's, so I personally have written as many words about it.

But I have the same concerns about all marketing that positions cars as being capable enough that people don't have to pay attention. Until we hit true Level 4 or Level 5 self driving cards, I'm extremely concerned about the public's perception that they don't have to operate a motor vehicle. *Especially* if the vehicle is capable of enough automated driving that the driver doesn't need to pay attention during most of the operation of the vehicle.

To your specific point though, I think in general the public would view something marketed at "autopilot" as more capable than something marketed as "co-pilot", despite the significant capability advantage that a "co-pilot" actually offers over an "autopilot" inside the cockpit of an airplane.

The difference between Ford and Tesla is that Ford is "one of the good ones".

Not interested in leading the pack; happy enough to follow, crush smaller competition by their bloat, declare bankruptcy and take in hundreds of billions in handouts because of their irresponsibility and ineptitude.

Maybe that's the difference for other people, but it's not the difference for me.

Legacy branding doesn't mean anything when it comes to implementing and marketing self driving vehicles.

I care about the quality and level of the implementation, and the marketing positioning.

Because unlike auto-pilot the implication of co-pilot is that you're still primarily in charge of flying/driving/piloting the vehicle.

It’s not, the co-pilot might fly the plane for most of the flight.

Umm no. Here's how it works

me: "you have the controls"

co-pilot: "I have the controls"

me: "you have the controls"

I am no longer flying the aircraft or even paying much attention to it. I could even get up and walk away from the controls. That's why co-pilots exist

The response I had earlier in the thread is absolutely applicable to this.

It doesn't matter. At. All. How the operations work inside the cockpit of an airplane. What actually happens in an airplane is 0% important to the conversation.

What actually matters is the public at large's perception of the term "autopilot" and their perception of the term "co-pilot". If the public perceives "co-pilot" as being less functional than "autopilot" then it does not matter at all that in aviation a co-pilot is actually more capable than an autopilot.

That would imply that people know aircraft-slang. Which they clearly don't.

Common perception of a co-pilot is that a co-pilot _assists_ the pilot.

It's another perception/reality thing. I realize that the co-pilot shares the load, and is capable of being the pilot-in-charge, or having control of the aircraft. But one implies self piloting (Tesla), the other implies an intelligent assistant pilot.

I think it's because of the general nature of the prefixes of the words.

Co-pilot says that the two of you are doing it together. Co-operatively.

Auto = automatic = you no longer have to do it.

Think of an automatic vs. stick shift car. And automatic car you no longer have to shift.

Autopilot saves lives even in current state.

Based on what?

What could save lives is intelligent automatic breaking - because that is something we are currently capable of. Humans as a species is terrible at paying attention to boring tasks and quickly react - so these gimmick autopilot takes are dangerous if anything.

It does that as well. Have you ever driven one?

Most standard cars already come with emergency automatic breaking.

Is that so? Now if they also could brake...

That's exactly what self driving cars do, silly.

They do it as well on top of some other non-safe features. Less is sometimes more. Breaking when a pedestrian steps before us in sub-human reaction time is great - it doesn’t have to be combined with full self driving, which is simply really far away from now.

Just like tobaco is forced to print "this leads to cancer" on their products, self driving cars should have labels like "car crashing into wall" and "car killing other human".

Please link to reports of a Tesla running over someone in AP.

Let's not forget that Tesla is well aware the car doesn't "drive itself" and have classified AP/FSD for legal/liability reasons as a level 2/3 system.

From a marketing perspective however, robotaxis are just around the corner.

The average consumer is not familiar enough with the limits of automated avionics to understand the limits of commercial jets' autopilot.

If the average consumer was also an airline pilot then it would make sense to use an industry term with nuanced meaning. Instead, the average consumer assumes a straightforward meaning of the term, especially with pop culture throwing out phrases like "planes practically fly themselves these days".

Claiming "autopilot is technically correct" would only approach being true if Tesla & others went into great detail to educate buyers about the limits of the avionic autopilot features referenced by the term, and difficulties of flying a plane in non-optimal conditions, so they actually understood the "technical" meaning and could apply it correctly to the car they're buying.

They tell you every time you engage it the limitations. Stop this fud. It is better than humans being in control.

I can't set target altitude nor heading on this thing, it can't use ILS, follow VORs. It doesn't make sense to call it autopilot

The passengers have no idea there are no pilots around when things don’t go as expected

Remember how dangerous the roads were when the car companies producing cars without gear shifts started calling their cars "automatic" as opposed to manual?

This is ridiculous. A transmission failure and a self-driving failure have very different implications. Automatic transmissions don't have pedestrians walking through them, etc.

You say this is ridiculous but your premise that these new "self-driving" cars are "walking trough pedestrians" is even more ridiculous and not backed by any data of real world experiments currently being undertaken.

I think the point was that automatic transmissions are far simpler systems that are harder to screw up. And if you do screw them up it’s bad but not driving into pedestrians bad.

No, there’s not many self driving cars that have driven into pedestrians but Uber ATG’s car notably did, and there is certainly a lot of room for failure.

Meanwhile, humans are driving into pedestrians every minute of every day, but yes, get upset about what Tesla called their system...

Reply on news.ycombinator.com