The fact that an huge formal investigation happened with just a single casualty is proof that it may actually be superior for safety in the long-term (when combined with feedback from regulators and government investigators). One death in conventional vehicles is irrelevant. But because of the high profile of Tesla’s technology, it garners a bunch of attention from the public and therefore regulators. This is PRECISELY the dynamic that led to the ridiculously safe airline record. The safer it is, the more that rare deaths will be investigated and the causes sussed out and fixed by industry and regulators together.
Perhaps industry/Tesla/whoever hates the regulators and investigations. But I think they are precisely what will cause self driving to become ever safer, and eventually become as safe as industry/Tesla claims, safer than human drivers while also being cheap and ubiquitous. Just like airline travel today. A remarkable combination of safety and affordability.
This might be the only way to ever do it. I don’t think the airline industry could’ve ever gotten to current levels of safety by testing everything on closed airfields and over empty land for hundreds of millions of flight hours before they had sufficient statistics to be equal to today.
It can’t happen without regulators and enforcement, either.
If anything a driver monitoring system seems even better than the interlock system, for example you couldn't have your kids/friends blow for you to bypass it.
Go with safest drivers first.
I think the comparison should be Tesla with/without AI, not Tesla/not-Tesla; so roughly either x2 or x4 depending on what the other active safety features do.
It’s not nothing, but it’s much less than the current sales pitch — and the current sales pitch is itself the problem here, for many legislators.
This still isn't the correct comparison. Major selection bias with comparing miles with autopilot engaged to miles without it engaged, since autopilot cannot be engaged in all situations.
A better test would be to compare accidents in Tesla vehicles with the autopilot feature enabled (engaged or not) to accidents in Tesla vehicles with the autopilot feature disabled.
https://www.iihs.org/topics/fatality-statistics/detail/urban...
Yes it can. The only time it can't be activated is if there is no clearly marked center line.
The real test of this would be: of all Tesla vehicles, are the ones with autopilot enabled statistically safer or less safe than the ones without autopilot enabled?
Makes me laugh, especially with the “geeks are immune to marketing” trope that floats around here equally as regularly.
1. Autopilot only works on (or intended to work on) highways. But they are comparing their highway record to all accident records including city driving, where accident rate is far higher than highway driving.
2. They're also comparing with every vehicle in the United States including millions of older vehicles. Modern vehicles are built for higher safety and have a ton of active safety features (emergency braking, collision prevention etc). Older vehicles are much more prone to accidents and that skews the numbers.
The reality is Teslas are no safer than any other vehicles in its class ($40k+). Their safety report is purely marketing spin.
Would another way of saying this be that they are as safe as other vehicles in that class? And that therefore Autopilot is not more unsafe than driving those other cars?
We should really compare Autopilot with its competitors like GM’s Super Cruise or Ford’s Blue Cruise, both of which offer more capabilities than Autopilot. That will show if Tesla’s driver assist system is more or less safe than their competitors product.
The reason GM's Super Cruise got a higher rating by Consumer Reports was because CR didn't even test the capabilities that only Tesla had (such as automatic lane change and taking offramps/onramps). Also, the majority of the evaluation criteria weren't about capabilities. eg: "unresponsive driver", "clear when safe to use", and "keeping the driver engaged".[1]
1. https://www.consumerreports.org/car-safety/cadillac-super-cr...
AP has the luxury of being able to be turned off in less than ideal conditions. Human drivers can't do that.
It's the reason why only Tesla touts these numbers. They're inaccurate and misleading.
Asking someone to pay attention when they are not doing anything is unrealistic. I would be constantly bored / distracted. My wife would instantly fall asleep. Etc etc.
Conversely, when Waymo iterates and improves their own safety ratios by a significant amount, that evidently does not result in Tesla's improving in return.
Until it leads to something systemic, that then regulator mandates for all vehicles
Not the case at all between lidars and cameras.
Of course this is impossible in the real world. Nobody is going to buy a car that will randomly make its own decisions, that will pull the wheel from your hands ever time it thinks you are making an illegal lane change. Want safety? How about a Tesla that is electronically incapable of speeding. Good luck selling that one.
I really don't think that would give many data points, because all of the instances would be when a human fell asleep or wasn't paying attention.
Because when the human disagrees with the machine, the machine is usually the one making a mistake. It might prevent accidents, but it would also cause them, and you lose predictability in the process (you have to model the human and the machine).
Nothing more annoying then a car that thinks i don't know how to drive (warning beeps etc.).
That's almost exactly what my Honda does. Illegal (no signal) lane change results in a steering wheel shaker (and optional audio alert). And the car, when sensing an abrupt swerve which is interpreted as the vehicle leaving the roadway, attempts to correct that via steering and brake inputs.
But, I agree with your more general point - the human still needs to be primary. My Honda doesn't allow me to remove my hands from the steering wheel for more than a second or two. Tesla should be doing the same, as no current "autopilot" system is truly automatic.
To be clear, tying the warning to the signal isn't about preventing unsignaled lane changes, it's gauging driver intent (i.e. is he asleep and drifting or just trying to change lanes). It's just gravy that it will train bad drivers to use their signals properly.
> 142 (1) The driver or operator of a vehicle upon a highway before turning (...) from one lane for traffic to another lane for traffic (...) shall first see that the movement can be made in safety, and if the operation of any other vehicle may be affected by the movement shall give a signal plainly visible to the driver or operator of the other vehicle of the intention to make the movement. R.S.O. 1990, c. H.8, s. 142 (1).
That said there's zero cost to doing so regardless of whether other drivers are affected.
Usually, it makes you liable because it is illegal. CA law for instance reqires signalling 100ft before a lane change or turn.
By the way, this is fucking terrifying when you first encounter it in a rental car on a dark road with poor lane markings while just trying to get to your hotel after a five hour flight.
I didn't encounter an obvious wheel shaker, but this psychotic car was just yanking the wheel in different directions as I was trying to merge onto a highway.
Must be what a malfunctioning MCAS felt like in a 737 MAX, but thankfully without the hundreds of pounds of hydraulic force.
As a simple example, ABS (https://en.wikipedia.org/wiki/Anti-lock_braking_system) only interferes with what the driver does when an error occurs.
More related to self-driving, there’s various variants of https://en.wikipedia.org/wiki/Lane_departure_warning_system that do take control of the car.
And it is far from “incapable of speeding”, but BMW, Audi and Mercedes-Benz “voluntarily” and sort-of limit the speed of their cars to 250km/hour (https://www.autoevolution.com/news/gentlemens-agreement-not-...)
That would be unsafe in many situations. If the flow of traffic is substantially above the speed limit--which it often is--being unable to match it increases the risk of accident. This is known as the Solomon curve [1].
With the logic presented in the theoretical foundation section, it seems that the safer move would actually be slow down and match the speed of all the trucks and other large vehicles... which won't happen.
Matching speed sounds great, except there are always people willing to go faster and faster. In my state they raised the speed limit from 70 to 75, it just means more people are going 85-90. How is that safer?
However, you individually going 70-75 when everyone else is going 85-90 is less safe than you going 85-90 like everyone else in the exact same situation.
>there are always people willing to go faster and faster
That’s why no one says “go as fast as the fastest vehicle you see”, it is “go with the general speed of traffic”. That’s an exercise for human judgement to figure that one out, which is why imo it isn’t a smart idea to have the car automatically lock you out of overriding the speed limit.
And yet the roads are full of vehicles literally incapable of going 85. Many trucks cannot do more than 69mph.
(Trivial if the self driving tech works at all....)
There's just too much shit that can't be "seen" with a camera/sensor in conjested traffic. Having a swarm of vehicles all gathering/sharing data is one of the only true ways forward IMO.
My insurance company gives a lower rate if you buy the full autopilot option, and that to me indicates they agree it drives better than I, or other humans, do.
If following the speed limit makes cars safer, another way to achieve that without autopilot is to just have all cars limit their speed to the speed limit.
Sometimes it’s wonky when the street lines are unclear. It’s not perfect but a better driver than I am in 80% of cases
The problem is in those 20% of cases where you'd lulled into boredom by autopilot as you concentrate on designing your next project in your head, then suddenly autopilot says "I lost track of where the road is, here you do it!" and you have to quickly gain context and figure out what the right thing to do is.
Some autopilot systems use eye tracking to make sure that the driver is at least looking at the road, but that doesn't guarantee that he's paying attention. But at least that's harder to defeat than Tesla's "nudge the steering wheel once in a while" method.
The devil is in the details... GPS may not provide sufficient resolution. Construction zones. School zones with variable hours. Tunnels. Adverse road conditions. Changes to the underlying roads. Different classes of vehicles. Etc.
By the time you account for all the mapping and/or perception, you could've just improved the autonomous driving and eliminated the biggest source of humans driving: The human.
The parent post said that autopilot won't let him go over the speed limit and implies that makes him safer. My point is that you don't need full autopilot for that.
So this is not a technical problem at all, but a political one. As the past year has shown, people won't put up with any convenience or restriction, even if it could save lives (not even if it could save thousands of lives)
What would be even easier than all of that, though, is just installing speeding cameras and mailing tickets.
I have not yet struggled to stay alert when it drives me, and it has driven better than I would have - so it certainly is an improvement over me driving 100% of the time. It does not have road rage and it does not enjoy the feeling of speeding, like I do when I drive, nor does it feel like driving is a competition, like I must admit I do when I am hungry, stressed, or tired.
> just have all cars limit their speed to the speed limit
No way I’d buy a car that does not accelerate when I hit the pedal. Would you buy a machine that is not your servant?
I mean... that's an odd thing for someone to say who has bought a vehicle with over-the-air firmware updates.
I wish I could hack but car, but also wouldn’t trust others if they did
>>How about a Tesla that is electronically incapable of speeding. Good luck selling that one.
From 2022 all cars sold in the EU have to have an electronic limiter that keeps you to the posted speed limit(by cutting power if you are already going faster) - the regulation does allow the system to be temporarily disabled however.
I think it's a silly law and I'm very glad I don't live in a place that requires such annoyances, but it's not as bad as you're claiming.
1. https://etsc.eu/briefing-intelligent-speed-assistance-isa/
It could reliably recognize the speed limit signs (red circle), but it never recognized the similar grey-slash end-of-limit signs. It also didn't recognize the start-of-town or end-of-town signs, so it didn't do anything about the limits they implied.
I would certainly have had to disable it, had it been reducing the acceleration in the way that document describes.
Yeah, add to that the unreliability of Tesla's system means that it cannot pull the wheel from the driver, because it's not unusual for it to want to do something dangerous and need to be stopped. You don't want it to "fix" a mistake by driving someone into the median divider.
Instead they did the exact opposite with the plaid mode model S, lol. It kind of works against their claims that they prioritize safety when their hottest new car - fully intended for public roads - has as its main selling point the ability to accelerate from 60-120 mph faster than any other car.
People literally are buying cars that “make their own decisions”. Importantly though, these systems only activate in the case of an imminent collision IF the corrective measure won’t cause another collision.
> that will pull the wheel …
Yah, of course no one is going to buy a care that does what you describe because what you describe is insane and inherently unsafe. Unless a collision is imminent, nothing happens.
I need to be touching the wheel and applying some force to it or it begins yelling at me and eventually brings me slowly to a stop.
I’ve had it for a year now and I cannot perceive of a way, without physically altering the system (like hanging a weight from the wheel maybe?) that would allow me to stop being an active participant.
I think the opposite is true: Tesla’s move fast and kill people approach is the mistake. Incremental mastering of autonomous capabilities is the way to go.
> physically altering the system (like hanging a weight from the wheel maybe?)
was exactly what people were doing. But it's also possible to be physically present, applying force, but being "zoned out", even without malicious intent.
I've occasionally noticed myself zoning out behind the wheel of my non-self-driving car as well.
It's actually very common. [0]
[0] https://www.actuarialpost.co.uk/article/quarter-of-fatal-cra...
> I’ve had it for a year now and I cannot perceive of a way, without physically altering the system (like hanging a weight from the wheel maybe?) that would allow me to stop being an active participant.
That's exactly what people were doing with the Tesla. Hanging a weight to ensure the safety system doesn't kick in. [0][1]
[0] https://edition.cnn.com/2021/04/28/cars/tesla-texas-crash-au...
[1] https://twitter.com/ItsKimJava/status/1388240600491859968/ph...
I guess the probe will reveal what share of fatal accidents are caused by this.
Teslas can famously be tricked by wedging an orange between the rim and spoke of the steering wheel to produce enough torque on the wheel to satisfy the detection. There are enough videos of it on youtibe that tesla could easily be found negligent for not doing enough to prevent drivers from defeating a safety system given that alternate technology that more directly tracks attention is available and tricking tesla's detection method became common knowledge.
Personally Autopilot has actually made driving safer for me... I think there's likely abuse of the system though that Tesla could work harder to prevent.
They are sending multiple signals that this car can drive itself (going so far as charging people money explicitly for the "self-driving" feature) when it cannot in the slightest do much more than stay straight on an empty highway.
They should be forced to change the name of the self-driving features, I personally think "Backseat Driver" would be more appropriate.
It is literally an autopilot. Just like an autopilot on an airplane, it keeps you stable and in a certain flight corridor. There's virtually no difference except for Tesla's Autopilot's need to deal with curved trajectories.
Well, and it actively avoids collisions with other vehicles (most of the time). Airplane (and boat) autopilots don't do that.
"But you're using the word autopilot wrong!"
Damn, this 'drink verification can' is going to get us all killed.
People causing these problems almost certainly are putting something over the cabin camera and a defeat device on the steering wheel.
It therefore isn't a clean swap of a human paying attention to a human who isn't. It becomes a complicated equation that we can't just dismiss with "people won't pay attention". It is possible that a 90%/10% split of drivers paying attention to not paying attention is more dangerous when they are all driving manually than a 70%/30% split if those drivers are all using self-driving tech to cover for them. Wouldn't you feel safer if the driver behind you who is answering texts was using this incremental self-driving tech rather than driving manually?
No one has enough data on the performance of these systems or how the population of drivers use them to say definitively that they are either safer or more dangerous on the whole. But it is definitely something that needs to be investigated and researched.
That is the fatal flaw in anything but a perfect system - any kind of system that is taking the decisions about steering from the driver is going to result in the driver at best thinking about other things and worse getting into the back seat to change. If you had to develop a system to make sure someone was paying attention, you wouldn't make them sit in a warm comfy seat looking at a screen - you would make them actively engage with what they were looking at - like steering.
And ultimately it doesn't matter how many hundreds of thousands of hours of driving you teach your system with, it may eventually be able to learn about parked cars, kerbs and road signs, but there won't be enough examples of different accidents and how emergency vehicles behave to ever make it behave safely. Humans can cope with driving emergencies fairly well (not perfectly admittedly) no matter how many they've been involved in using logic and higher level reasoning.
We've known for a very long time that this sort of automation/manual control handoff failure is a very big deal, and yet there seems to be an almost willful blindness from the manufacturers to address it in a meaningful way.
My car has adaptive cruise control and lane keep assist, but I'm not relying on either for anything more complex than sipping a drink while on the highway.
Conversely, if it didn't warn me right before an accident, then the absence of that warning would be useful too.
All of that information should be put back into the model based on crash reporting. Everything else can be ignored.
I would argue that the information should be available to all automakers (perhaps using the NHTSA as a conduit), so that each of them have the same safety information, but can still develop their own models. The FAA actually does this already with the FAA Accident and Incident Data Systems [0] and it has worked pretty darn well.
It also reads the speed limit signs and places a reminder in the display. I think it can brake if it detects something in front of it, but I'm not certain.
My main point (perhaps buried more than it should have been) is that by centralizing accident data along with whether an alert went off (or not), and sharing that with all automobile manufacturers can help this process proceed better.
Right now the data is highly fragmented and there is not really a common objective metric by which to make decisions to improve models.
I agree that would be reasonable and desirable :-)
These things are on the road already. They have issues, but so do human only cars. Tweaks probably get made, like some special handling of emergency vehicle scenarios. But, it's not enough to stop it.
Meanwhile, it's not a permanent state. Self driving technology is advancing, becoming more common on roads. Procedures, as well as infrastructure, is growing around the existence of self driven cars. Human supervisor or not, the way these things use the road affects the design of roads. If your emergency speed sign isn't being headed by self driven cars, your emergency speed sign has a bug.
"The average driver" includes everyone, ranging from drivers using it as intended with close supervision, drivers who become inattentive because nothing is happening, and drivers who think it's a reasonable idea to climb into the back seat with a water bottle duct taped to the steering wheel to bypass the sensor.
OTOH, the average driver for the unassisted scenario also includes the driver who thinks they're able to drive a car while texting.
Shouldn't that compared to "average driver + myriad of modern little safety features" instead of "average unassisted driver"? The one who has the means to drive a Tesla with the "full driving" mode certain has the means to buy, say, a Toyota full of assistance/safety features (lane change assist, unwanted lane change warning and whatnots).
Making it a crime isn't an "obvious solution" to actually make it not happen. Drunk driving is a crime and yet people keep doing it. Same with texting and driving.
Prevention as a goal is how we end up with dystopia.
I'm not saying improvements should stop there, but once the system has reached parity, it's OK to deploy it and let it improve from there.
An imperfect self driving vehicle is the worst of all worlds: they lull the driver into the perception that the vehicle is safe while not being able to handle abnormal situations. The fact that there are multiple crashes on the record where Telsas have driven into stationary trucks and obstacles on roads is pretty damning proof that drivers can't always react in the time required when an imperfect self driving system is in use. They're not intrinsically safe.
At the very least drivers should be required additional training to operate these systems. Like pilots, drivers need to be taught how to recognize when things go awry and react to possible failures. Anything less is not rooted in safety culture, and it's good to see there are at least a few people starting to shine the light on how these systems are being implemented from a safety perspective.
Nothing absurd about thinking a system which has parity with the average human driver is too risky to buy unless you consider yourself to be below average at driving. (As it is, most people consider themselves to be better than average drivers, and some of them are even right!) The accident statistics that comprise the "average human accident rate" are also disproportionately caused by humans you'd try to discourage from driving in those circumstances...
Another very obvious problem is that an automated system which kills at the same rate per mile as an average human drivers will tend to be driven a lot more because no effort (and probably replace better-than-average commercial drivers long before teenagers and occasional-but-disproportionately-deadly drivers can afford it).
Doesn't this hide a paradox? Using a self-driving car as intended implies that the driver relinquishes a part of the human decision making process to the car. While close supervision implies that the driver can always take control back from the car, and therefore carries full personal responsibility of what happens.
The caveat here is that the car might make decisions in a rapidly changing, complex context which the driver might disagree with, but has no time to correct for through manual intervention. e.g. hitting a cyclist because the autonomous system made an erroneous assertion.
Here's another way of looking at this: if you're in a self-driving car, are you a passenger or a driver? Do you intend to drive the car yourself or let the car transport you to your destination?
In the unassisted scenario, it's clear that both intentions are one and the same. If you want to get to your location, you can't but drive the car yourself. Therefore you can't but assume full personal responsibility for your driving. Can the same be said about a vehicle that's specifically designed and marketed as "self-driving" and "autonomous"?
As a driver, you don't just relinquish part of the decision making process to the car, what essentially happens is that you put your trust in how the machine learning processes that steer the car were taught to perceive the world by their manufacturer. So, if both car and occupant disagree and the ensuing result is an accident, who's at fault? The car? The occupant? The manufacturer? Or the person seeking damages because their dog ended up wounded?
The issue here isn't that self-driving cars are inherently more dangerous then their "dumb" counter parts. It's that driving a self-driving car creates it's own separate class of liabilities and questions regarding responsible driving when accidents do happen.
Tesla auto-drive seems like it's about 80% of the way there.
It's just that the companies that are NOT doing incremental approaches are largely at the mercy of some investors who don't know a thing about self-driving, and they may die at any time.
I agree with you that it is technically flawed, but it may still be viable in the end. At least their existence is not dependent on the mercy of some fools who don't get it, they just sell cars to stay alive.
That's one of the major problems of today's version of capitalism -- it encourages technically flawed ways to achieve scientific advancement.
It is not solely the trust and dependence but inclusive is the group of idiots with access to wealth without regard to human life.
I will be amused, intrigued, and possibly a bit horrified if by the time FSD hits level 5, and they stick with the Neural Net of Neural Nets architecture if there isn't a rash of system induced variance in behavior as emergent phenomena take shape.
Imagined news: All Tesla's on I-95 engaged in creating patterns whereby all non-Teala traffic was bordered by a Tesla on each side. Almost like a game of Go, says expert. Researchers stumped.
Then again, that's imply you had an NN capable of retraining itself on the fly to some limited degree, which I assume no one sane would put into service... Hopefully this comment doesn't suffer a date of not aging well.
[0] https://theoatmeal.com/blog/google_self_driving_car
Car traffic and streets are more dense and often have humans crossing them without regards to laws, bicycles, motorbikes, road construction and bad weather.
Not saying one auto pilot system is better than the other, however, they operate in different environments.
P.S. /s. Obviously, Mr. Poe.
This is why 'attention' and 'driver monitoring' was not included.
It's okay. I do it too. Really need to work on seeing yourself making that argument as a starting point and not an endpoint.