Tesla’s Autopilot was engaged when Model 3 crashed into truck, report states
theverge.com
theverge.com
This is nonsense. The whole core of the argument against this sort of system is that it induces inattention. The fact that Tesla consistently refuses to share full data on identical models driving with autopilot turned on, enabled but not turned on, and not enabled is extremely suggestive that the overall safety rates for autopilot on are not better.
(edited for formatting)
Still we as a society usually decide not ban things that hurt only irresponsible people. There is an exception to this for children, but they are banned from driving already.
If the autopilot crashes start resulting in a number of deaths outside of the drivers car that might shift the public opinion.
> In the previous eight years, 6,100 people had been sent to the emergency room due to lawn darts in the U.S. Out of that total, 81% were 15 or younger, and half of them were 10 or younger. On the week the commission voted to ban the product, an 11-year-old girl in Tennessee was hit by a lawn dart and sent into a coma.
From wikipedia. It's a bad example.
EDIT: The accident rate is a statistic thrown by Tesla and only serves the purpose its meant to serve. Do we have any statistics on accidents caused comparing the situation when car is on auto-pilot vs similar cars driven by similar drivers under similar conditions without autopilot ?
Autopilot is getting worse not better with time, and Tesla's tendency to introduce regressions (as with the Bay Area crash) means any temporary improvements in driving AI are just that--temporary.
By a subset of the driving population who are less risky than the overall driving population.
This number by itself is absolutely meaningless without more context.
In 2017, 37,133 people in the United States died in an automotive crash. It is expected that some number of Tesla owners will die in a crash, regardless of currently engaged safety features.
Tesla has produced around 600,000 vehicles in total, including over 100,000 Model 3 vehicles.
I'm not making any claim or dispute regarding the effectiveness of Autopilot, whether or not it's improving. I'm just saying, making that statement you made is meaningless at least and misleading at most.
I'm just pulling a page from Elon Musk and Tesla's marketing department.
Also, do you have any proof that Autopilot is getting worse over time that doesn't rely on anecdotes?
EDIT: (I'm assuming for the purposes of this statement that you are correct and those were easily avoidable by an attentive driver)
[1] https://www.latimes.com/business/la-fi-tesla-autopilot-claim...
Tesla's has never claimed that a current vehicle can drive itself without the driver being responsible for the vehicle at all times.
> Tesla's has never claimed that a current vehicle can drive itself without the driver being responsible for the vehicle at all times.
Yes, they did. Hence why the NHTSA and other groups went after Tesla for their claims about Autopilot.
The Tesla autopilot controls the throttle, brake, and steering. That meets my definition of "driving."
It routinely false positives on frontal collisions and occasionally does things that would be dangerous if I weren't driving.
The driver in a Tesla is not a mere passenger, regardless of whether or not they have enabled autopilot. The initial warnings on enabling autopilot as well as the attention nags on the steering wheel all make this perfectly clear to the driver.
If, as Tesla insists, these autopilot crashes are all due to the driver not paying attention, then it seems very plausible that a better attention-detecting system (one that tracked the driver's gaze, for example) would improve safety. Of course, if the system required drivers to do what Tesla says they should be doing, it would not be so appealing.
And if, as Musk is saying, drivers are confused or mistaken about the capabilities of the vehicle, then the first response should be to stop making ambivalent statements about them.
It's why people freak out about terrorism more than they do about car crashes or drug addicition.
Tesla attached a lot more caveats than you did in order to assert superior safety.
Emphasis on reliable, as Tesla's own stats are highly misleading -- they basically compare highway data, because Autopilot is mostly used on highways, to all other conditions in which accidents happen.
I'm sure that if Autopilot was active all the time, the rate of accidents would be far higher, and Tesla's numbers wouldn't look so positive.
Say you stumble on a lost civilization in the woods, and they have yet to discover any form of shelter. Any time it rains or snows they all sit on the ground and half of them die of pneumonia. What do we, as engineers, do to help them? Tie some sticks together and make a lean-to? That would certainly reduce the death rate. But I don't think it would be the ethical thing to do. The ethical thing would be to help them build proper buildings out of the best available materials, install an HVAC system, fire alarms, etc.
In other words, engineering should seek to maximize benefits for users, and there are cases where choosing not to maximize benefits could be seen as unethical, even if you do provide some benefit.
In the case of Tesla, the ethical thing to do would be to include a lidar which can prevent the car from smashing into large objects at full speed.
Just because the accident rate is lower does not mean this death or all the others are acceptable. Tesla had a choice of whether to use best engineering practices, and they chose not do.
The reason they are doing this is money. They couldn't sell cars with an expensive lidar on it. And they shot themselves in the foot by charging people thousands of dollars for future "full self driving" capability. Now if they don't continue forward with this plan, with no lidar, it will have serious financial consequences for them.
Note that other self-driving companies have found a way to make lidar financially viable for them. It's not impossible.
Tesla is killing people for money.
There are fair criticisms around Tesla's marketing and implementation, but I don't think "they should already have lidar on there" is really one of them.
Which sounded pretty harsh to me at first, but when you think about it, thats effectively the same as the HN narrative: "yes things like the PATRIOT act could reduce terrorism, but it comes at the expense of our American freedoms, and a couple of lives simply isn't worth it."
And now for these cars: "yes removing Autopilot could reduce car crashes, but it comes at the expense of our American (?) freedoms (maybe freedom to drive/technically innovate/something else entirely), and a couple of lives simply isn't worth it."
One additional trend I haven't personally witnessed, but I hear about, is that when in some freak accident "a couple of lives" includes the life of someone who one of these gun guys, PATRIOT guys, or car guys knows, their opinion flips.
Benjamin Franklin has a particularly relevant quote on this subject.
We did in '34, '68 and '86. And then there's all the states that are still waiting for SCOTUS to tell them what "shall not be infringed" means.
In my state I have to pay hundreds of dollars and rely on the discretion of the local police in order to bear any semi-modern arms and that's just the tip of the iceberg. If that's not infringement I don't know what is.
https://www.lawfareblog.com/what-ben-franklin-really-said#.U...
There is no freedom from danger or "freedom to be safe". Therefore one party has sacrificed something of value to placate a sentimental imagining that amounts to nothing to the other side. That isn't a trade. That's loss. No agency is gained by that arrangement. It's the equivalent of legislative/rhetorical marketing language. Sounds nice, but ultimately empty.
Safety is a function of the inhabitants of an environment's ability to mitigate threats. If the environment is full of disarmed people, then the person who brings the gun anyway faces nil resistance.
No one is safe. No one has agency to counter the threat, and there is nothing to show for the sacrifice of the right except the ill will of those who recognize the physical, realistic (read: pragmatic) implication of the act, and disagree in the first place. Which, ironically, decreases safety even further.
Tech like Autopilot is actually kind of the reverse of the gun debate. It's an active development that increases complexity creates new problems, and decreases agency since everything then gets bound by "Autopilot safety is a must." The argument is made that automating the task with a machine must make the thing safer, because every act of automating things up to now through a machine has made things safer, which is just an inductive fallacy if I ever heard one.
Sometimes the things you don't do mean as much if not more than the things you do. A point oft overlooked, and lamented by the "Do as I say not as I do" parent.
Anyway. That was a fun tangent.
The freedom to walk down the street without risk of being shot is not sentimental. Presenting it as so is disingenuous.
Safety is a function of the inhabitants of an environment's ability to mitigate threats.
The evidence shows that isn't working.
> The evidence shows that isn't working.
No, the evidence actually shows that that does work - gun free zones are home to something like 99% of all mass shootings, and school shootings with a prepared police officer or teacher carrying a weapon tend not to make the news because they don't hit the level of 'mass shooting' in the first place.
And then there's the debate about whether forcing terrorists to choose a methodology other than guns is even a good idea in the first place - I'd rather have ten mass shootings killing ~4-6 people each than a single repeat of Oklahoma City.
On the collective front for example, the anti-arms crowd sees a citizen exercising a right as a threat. The pro-arms side sees the anti-arms's willingness to engage in rhetoric to dismantle a fundamental right as a threat to the overall underpinnings of civil society. The pro-arms is not innately threatened by the bearing of arms, or refusal to do so, but by enforced asymmetric eligibility to bear arms without a darn good reason. This is the basic blueprint for any major X/!X or an X/!(X&&Y) divide [I think I got the truth table right there].
On the individual front, things get even more diverse.
A gang member sees a rival gang member on his turf as a threat, but the "civie" right next to them sees neither as a threat until the violence starts, and they have no defense.
A harassed/abused/ostracized child generalized the infliction of suffering by members of their peer group to the rest of humanity, making everyone a threat.
An oppressed/marginalized population feels threatened by their oppressors.
A previously oppressed/marginalized populations feels threatened by their previous oppressors
That's what I mean by there is no positive "agency" or "essence" to the "freedom to be safe". The state of safety is accidental to the circumstance. As long as there is an agent willing to break any civil consensus in such a way as to cause harm, the best move is for everyone to possess the most effective means of causing harm, but to refrain from employing it. This protects and perpetuates civil society, without creating "soft target" situations where no resistance can be offered to a violator of the peace.
I respect your right to the view that there is some positive existence to the freedom to be safe. However, at most I'll acknowledge the legitimacy of a claim to the right to live a life in the pursuit of Safety to be semantically and grammatically capable of bearing significance and recognizable meaning; in parity with the recognition of the purpose of a government to secure for its' People the right to Life, Liberty. and the pursuit of Happiness as proscribed to in the United States.
Pursue, by all means, but finding/catching it is far from guaranteed, and depriving others of what brings them Safety will inevitably set the stage for violent conflict, even when done through civil means.
However, I digress, and fear this interesting diversion may have long overstayed it's welcome, and it certainly was not my intent to turn this thread so far off topic. I just had a moment of clarity enabling me to formulate an articulation of an observation that has hitherto resisted my attempts to elucidate.
Having said that, this case is a bit special, IMO. By your own logic, this comes down to a cold calculation of how many crashes (and deaths) are acceptable. To the regulating agencies, the # is likely relative to other vehicles regardless of AutoPilot. So if AutoPilot is not demonstrably less safe than the others, why regulate it out of existence?
So far, such evidence simply doesn't exist. And honestly, the Tesla effect magnifying every incident doesn't make it better. Tesla is far from the only vehicle with systems like this on the road.
"I told you so" cuts it for tens of thousands of deaths caused by user error per year.
I'm not saying you're wrong about Tesla's autopilot, but it should be put in proper context. Tens of millions of lightly trained people are controlling thousand-pound steel projectiles at speeds where stopping distances are measured in hundreds of feet. Gruesome results are par for the course. I know we're all hoping that these deep-pocketed companies with fancy tech and big research budgets are going to fix it all for us, but they're not going to solve it on their first go.
I do get your point regarding cognitive fatigue though but I think we need a conclusive statistic rather than one thrown by Tesla.
“There was sufficient sight distance to afford time for either the truck driver or the car driver to have acted to prevent the crash.”
“The Tesla’s automated vehicle control system was not designed to, and did not, identify the truck crossing the car’s path or recognize the impending crash; consequently, the Autopilot system did not reduce the car’s velocity, the forward collision warning system did not provide an alert, and the automatic emergency braking did not activate.”
“If automated vehicle control systems do not automatically restrict their own operation to those conditions for which they were designed and are appropriate, the risk of driver misuse remains.”
“The Tesla driver was not attentive to the driving task, but investigators could not determine from the available evidence the reason for his inattention.”
https://www.ntsb.gov/investigations/AccidentReports/Reports/...
I am not a domain expert. Front facing radar and cameras alone is clearly inadequate for forward facing object detection [1] (unless we're going to mandate radar detectable skirts and images superimposed on the sides of tractor trailers for vehicle detection). Would solid state LIDAR (essentially laser ranging I suppose, as you're not building a 360 degree point cloud) solve this particular edge case? Some automakers are including laser headlights in their vehicles [2] (although I don't believe this is yet approved for the US market); would it not make sense to convert the headlights into front laser ranging sensing systems along with their illumination function?
[1] https://www.caranddriver.com/features/a24511826/safety-featu...
[2] https://www.osram.com/am/specials/trends-in-automotive-light...
The Neural Network should be capable to recognize a truck from any angle and even if parts are obstructed. I hope we will get the details public and see if it actually tagged the obstacle but ignored it or it detect it as something else like maybe a side road advertising panel
One of these crashes involved a firetruck, which unlike a semi-trailer, does not have a large gap underneath it.
Radars are limited period. The radars used by Tesla lack the angular resolution to distinguish between a stationary object next to the road (e.g. a building or parked car) with a stationary object parked right in the middle of the road.
The radar that companies people on HN don't think highly of sell to the government is "scary good" and that's just the decade+ old tech that I'm familiar with.
I'm sure there's reasons Tesla doesn't/can't put that in cars though.
Elon caving and adopting Lidar is probably more likely.
That fire truck wasn't crossing the road, it was at the road side, which creates a different situation. The problem with high trailers is, that they very much look like a bridge to a radar. In any case, a proper side bareer can prevent deadly accidents.
https://twitter.com/CC_Firefighters/status/95552999131956019...
It's insufficient angular resolution for the application, no matter which way you slice it.
On the other hand, the engagement of autopilot might have been caused by a desire to explicitly NOT pay attention, however briefly. think: "ok, now let me read this text from the person I'm on my way to meet."
I have found myself occasionally using autopilot to augment my driving when I'm doing something that inherently decreases focus on the road, however briefly; eg, taking my sunglasses out of the glovebox. I still drive the same way I normally would in any other car when reaching for something in the glovebox, so it's easy to think "any enhanced safety is better than none".
On the other hand, it could lead to a false sense of security where you pay 10% less attention than doing the same task in a non-augmented car. Or you get comfortable doing such things, so you do them more frequently than truly necessary. (for example, you might just shield your eyes from the sun with your hand rather than grabbing sunglasses in a non-augmented car).
The issue isn't with the sensors, but rather the system reacting to sensor data.
The stuff the DoD buys (or was buying a decade ago) is good enough to tell two basically identical tractor trailers apart based on minor features, dents/dings, etc.
As the long slow march of progress continues it may come down in cost enough to be viable on vehicles.
As I understand it it's the algorithms that analyze the radar data that are the secret (literally) sauce. I didn't work on any of that. I just plumbed various data from A to B. I don't know much other than it was really damned good for being "legacy" at the time I was working there.
So yes.
This seems to be a hard problem where you don't want the model to false positive on a stationary object on the side of the road and slam on the brakes.
So if I understand you properly, an adversarial attack against Tesla autopilots would be to suspend a ladder across the road at windshield height?
"Vulnerable to being clotheslined" seems like a bit of an oversight.