Tesla on autopilot crashes into stopped Michigan police car
autonews.com
autonews.com
Well, one can argue that someone willing to drive after his license is revoked is also willing to lie about the Tesla crashing itself. It seems super reasonable to wait for Tesla side instead of rushing to conclusions.
https://apnews.com/article/legislature-legislation-local-gov...
Highlights:
> The company, for nearly 200 years of its existence (1602–1800), had effectively transformed itself from a corporate entity into a state or an empire in its own right.
> With its pioneering institutional innovations and powerful roles in world history, the company is considered by many to be the first major, first modern, first global, most valuable, and most influential corporation ever seen.
William Dalrymple has a tome of a book about the English East India Company, which I hope to check out someday soon: https://www.theguardian.com/books/2019/sep/11/anarchy-relent...
Jason Hickel is also an invaluable source: https://www.aljazeera.com/opinions/2018/12/19/how-britain-st...
> The Tesla's Autopilot driver-assist system was engaged when it struck the police car, a blue Dodge Charger with its emergency lights activated, police officials tweeted.
Or did I misunderstand and are you saying you don't believe the police report here?
I had assumed the reasonable thing would occur; the plates would expire and become suspended, a non-issue for a chassis sawz-alled to bits and sold for scrap.
Nope! Not in IL. While receiving a benign speeding ticket on a road-trip out of state, I learned my ostensibly valid DL was suspended, and what it feels like to be handcuffed, arrested, and processed at the station.
Why had IL suspended my DL? Well, I was forced to stay out-of-state and appear in court that week, eventually learning why IL had suspended my license to defend my innocence: the disassembled vehicle hadn't passed a smog test that year. They suspended the vehicle owner's DL for this nonsense. The judge in the state I had to defend myself in didn't even know what a smog test was. It was a clown show across the board, with my pocket footing the bill at every turn.
A suspended license is not a reliable indicator of anything other than bureaucratic fundraising by the state.
That expectation seems to be at odds with everything I as a non car owner remember from my parents selling their cars. As far as I can find there often is no explicit requirement to deregister the car/plates, but with the exact problem you ran into: Even if everything ended up suspended the registration would still point at you, with a very good chance that you end up liable.
> hadn't passed a smog test that year.
Among the top reasons for people driving around with an expired registration apparently. So the police found exactly what it expected when it saw that expired registration with your name on it.
Selling and letting registration lapse are two very different things.
Most states only criminalize driving a car with a expired registration. I have half a dozen cars with expired registration to my name and it's a non issue as long as I don't get pulled over driving them (which would be really hard to do since they either aren't complete enough to be drive-able or have long since been turned into tin cans).
What are you talking about? I wasn't driving a vehicle with expired tags, the vehicle in my name triggering the DL suspension had been DIY scrapped and the plates destroyed. I was pulled over for speeding, driving a friend's car on a road trip during my turn at the wheel.
Reading comprehension fail...
Can you point out where you wrote that in your previous comment?
What the hell else is this supposed to mean? That I both scrapped the car and was driving around with the plates for some reason?
Plus, wouldn't it be spelled "sawzall-ed" (though, more likely, just "sawzalled")?
Isn't one still fully responsible even when the car is in autopilot mode? I feel like blaming a Tesla for "crashing itself in autopilot" is a bit like saying a Mercedes "crashed itself" because you put it on cruise control / lane assist / collision detection mode so you could text from the driver seat or some such.
Same as any other situation. If one 737 Max crashes, Boeing may blame the pilots. But if another crashes, Boeing has to take some responsibility.
https://www.nhtsa.gov/sites/nhtsa.dot.gov/files/documents/un...
(I don't believe this group has a particular tendency to want to hit stationary objects, though.)
Fun fact: North Dakota has the most drivers with a suspended license at 7.75% and is followed by Ohio at 5.62% [2].
[1]: https://insurify.com/insights/cities-with-the-most-suspended... [2]: https://insurify.com/insights/the-10-states-with-the-most-su...
Namely, this user wrote:
To add even more detail, as there still appears to be a lot of confusion:
The radar systems in these vehicles send out a radio pulse in a broad approximate-cone forward. They get bounces back from everything that reflects radio in front of them. Distance from the object is calculated by time between pulse and response. Speed towards/away from the object is calculated from Doppler shift of the radio frequency.
There are two main things that these systems can't detect.
1. Speed of the object perpendicular to the direction of radio wave travel.
2. Location of the object within the approximate-cone the radio pulse travels in.
Note that thanks to the second, you can't calculate the first with higher-level object tracking, either.
So the data you get back is a list of (same-direction velocity component, distance) pairs. There's no way to distinguish between stationary objects in the road and stationary objects above the road, to the side of the road, or even the surface of the road itself.
Radar just doesn't provide the directional information necessary to handle obstacle detection safely.
Tesla may be trying to optimize costs and be aggressive as they have been in other areas, doing the whole MVP/iteration thing, but this is one place where that just isn't necessarily the greatest strategy. They should be aiming to be better than human drivers not just as-good-as, and the challenges and limitations of ultra complex software based systems development are hardly some new revelation. They should have been prepared for that too with more hardware backstop.
I suggest a better headline: a 22-year old man with a suspended license crashes into stopped Michigan police car.
And the fact we only hear about one of these kinds of crashes every few months essentially proves it's true.
Remember, more than 100 people are killed on the roads each and every day of the year in the USA. Dig into the stats, one Tesla crashing does not mean that overall Autopilot is not effective.
Regardless what your opinion is on the naming, autopilot is a form of cruise control and legally and operationally requires driver attention, 100% of the time.
I'd say the CEO advertises it otherwise.
https://www.wired.com/story/elon-musk-tesla-autopilot-60-min...
https://www.tesla.com/sites/default/files/images/autosteer-e...
A disclaimer, like a clickwrap TOS, is not enough to take away Tesla's culpability. You can't sign away your life.
I mean, imagine this in any other industry. CEO: "Our new FireSafety-O brand insulation is fire safe!" [Some houses catch fire] Company lawyers: "Well, didn't you read the post-purchase disclaimer that says it's made out of shredded newspaper and kerosene, and is only suitable for use in a non-oxidizing atmosphere? Ignore the CEO, he's deranged."
"Adaptive Cruise Control"
"Lane Keeping Assistance"
"Automatic Emergency Braking"
The same names (standardized names, really) used by other auto companies offering similar capabilities.
> Update on the coast to coast autopilot demo?
> Still on for end of year. Just software limited. Any Tesla car with HW2 (all cars built since Oct last year) will be able to do this.
Date of tweets: 2017, almost 4 years ago. They've been advertising "coast to coast autopilot" and "Full Self Driving" features for years now.
That's the thing: the features of this are are named "autopilot" and "full self driving". What do you expect people to do when they spend $5000 or $10,000 on "full self driving" ?
It's dishonest to dismiss this with "regardless of your opinion on the naming".
They've had this capability for decades now; it was mentioned in a 5th season Mythbusters episode (the one about talking a civilian into a landing).
In that case, the law has not yet decided.
https://arstechnica.com/gadgets/2021/03/judge-rules-5-billio...
But if a plane on autopilot crashed because the pilot wasn't paying attention, the pilot would be at 100% fault. That's the origin of the naming, even if it's confusing to a layperson. I'm not here to argue about whether they should call it this or not.
Expecting the same level of competency and understanding of tools from an airline pilot, and the every day driver who hasn't passed a driving test since they were 16 years old, is ludicrous at best.
To put it another way, if a driver suffers a seizure on the highway, and Autopilot was able to avoid all collisions and bring the driver safely to the side of the road, what do you think the headline should be?
“Driver suffers seizure on interstate, automatic driving system steers car to safety”
Or:
“Man successfully drives on interstate while having an incapacitating seizure”
"We sold you a car that drives, we never said it can drive uphill!"
Regardless of you opinion, autopilot kills people, because its not possible to suddenly take over from an automoua car that decided to kill you, in 50ms. We have mountains of evidence on reaction time
What a clown!
> Autopilots do not replace human operators. Instead, the autopilot assists the operator's control of the vehicle, allowing the operator to focus on broader aspects of operations (for example, monitoring the trajectory, weather and on-board systems).
When you are piloting a vehicle it should be razor sharp that you are holding a weapon in your hand that is deadlier then a handgun.
We built a fancy computer vision model to detect and track football and it turns out a bald referee in a sunny weather can trick our model to think his head is the ball. So I wouldn't be surprised to know there are some flaws in their models too.
It's way safer to have other sensors and don't rely only on cameras if the purpose is engaged autopilot that takes control from the driver away.
fake it til you make it
The most interesting part of this, is that this is a case that humans screw up on a pretty regular basis too. However we understand the human reason for doing so (lack of concentration), and understand that AIs don't suffer from the same fault. So when an AI screws it up we are still surprised.
This isn't a case where it makes any sense to try to extrapolate from training data and call it a day. You don't need any training to avoid hitting stationary objects.
Compare this comment from 2018: https://news.ycombinator.com/item?id=17015528
> The first job of automated driving is to drive only on flat surfaces. Doesn't matter why it's not flat. Then you worry about where to go. That's how the off-road DARPA Grand Challenge vehicles had to work. "Flat road" is a pure geometry thing. Not much AI needed.
Why use "AI" to solve a problem you don't have in the first place? This is the wrong tool for the wrong job.
(Compare also GPT-2 chess, in which a model trained on transcripts of chess games isn't able to stop itself from making illegal moves. The solution is straightforward - give your chess model a model of the rules of chess! And don't attempt to drive into spaces you cannot physically occupy.)
Is there a relevant difference between what's "training" for an CPU driver and what's training for a human driver, and if so, what?
Computers have problems humans don't too of course... that's why we get these articles about "ridiculous crash happened".
I repeat myself: "I can see arguments that the program shouldn't hit stationary arguments, but the ones I see apply to humans too with only minor changes." That humans have extra problems doesn't really matter, so long as humans require training to cope with those problems that are similar to those computers have.
For example, computers may use a vision device that sweeps across the surroundings and then use lots of postprocessing to build a complete scene. That's similar to what the human eye and brain do, except for the details of how the sweeping sweeps and how the postprocessing puts together a picture.
You do, because you need to correctly conclude that there is going to be a stationary object there with high probability. As it turns out, that's difficult to do perfectly.
You need to know that there is a stationary object in the first place, of a kind you care about (not a plastic bag). I.e. that it's not actually a moving object, an optical illusion, etc.
You also need to know that you are going to end up moving through it (it's not a parked car that you will avoid by following your lane).
That's... not trivial.
(going to be was probably poor phrasing on my part, sorry)
And "add some partial blockage by a vehicle in the lane, retrain the AI, and then redeploy it" isn't a proper answer... wouldn't that be a different model AI that would need to be validated again?
> And "add some partial blockage by a vehicle in the lane, retrain the AI, and then redeploy it" isn't a proper answer... wouldn't that be a different model AI that would need to be validated again?
You're the one adding unnecessary constraints here. If the validation process harms more than helps, you don't do it. In all likelyhood the correct validation process is something lightweight that can be done quickly for simple changes.
This sort of training process is likely to be a continuous process, they're always ingesting new data, trying to learn how to drive better from it (and keep up with how patterns in the world change), and therefore continually deploying new models. Because that makes driving safer overall. Not a "this bad thing happened so let's figure out how to put out a bugfix for it".
That is not true at all these days, not to mention that even a slight change in illumination will result in completely different input data, the way current AIs work.
In my opinion until this is somehow solved "full self driving" in presence of other human drivers sharing the road is just a dream.
> According to recent data taken from the AAA Foundation for Traffic Safety, approximately 12 percent of all interstate highway deaths result from shoulder accidents. This means that an estimated 600 people a year are killed—and thousands more are injured—while making emergency stops off the highway.
https://aaafoundation.org/wp-content/uploads/2017/12/Pedestr...
It's unfortunately fairly difficult to find stats on accidents scoped solely to vehicles rear ending emergency vehicles partially obstructing a travel lane. The stat I present is a proxy.
There is always risk. You can de-risk, but it's usually impossible de-risk entirely in regards to life safety. You are at risk of death on the road with human drivers, and that risk is well quantified (NHSTA, NTSB, private auto insurance industry). Arguably, there is data to demonstrate that safety systems and lane keeping are safer when edge cases don't occur [1]. The question is: are those edge cases acceptable if the risk is reduced overall? Planes crash, but people still fly commercial. Again, we don't ban commercial air travel, but regulators and industry work together to ensure there are robust systems in place to remove as much risk as possible.
Should we ban the production and sale of lettuce too because a few people get sick or die [2]? Because it sure does sound like we say, “oh well, caveat emptor, maybe start cooking your lettuce instead of eating it in salads." (The same number of people have died from lettuce as from Tesla's Autopilot in roughly the same window of time [a decade, lumping together AP1, AP2, AP2.5, and FSD beta])
[1] https://www.tesla.com/VehicleSafetyReport
[2] https://www.buzzfeednews.com/article/venessawong/the-decade-... ("2010s total: More than 1,010 people sickened, including 6 deaths.")
Not if they keep hitting parked cars they won't, the law will catch up to them.
How many deaths in total have occurred related to Autopilot use? 6 [1]. About 100 people per day die in auto-related incidents. If you want to take it to the extreme, Tesla has killed far fewer people than Boeing's 737MAX autopilot (which has killed 346 people in a six month span).
>> Not if they keep hitting parked cars they won't, the law will catch up to them.
> Uber killed a person with a safety driver not paying attention and all safety systems turned off. Did the law change? It did not.
Hmmm. You seem to have glossed over the verb aspect. How many people did Uber kill like that?
The link you posted shows a clear correlation between low light conditions and hitting people on the shoulder. A police cruiser has blue lights and presumable the federally mandated reflective surfaces that all vehicles sold in the US have (usually in the tail lights).
This is a typical case of software making stupid mistakes a human would never make.
Humans making this mistake do tend to brake at some point, just too late. There's still an accident, but damage is mitigated and survival odds, with seat belts and airbags, are good. Teslas are notorious for plowing into unrecognized stationary obstacles at full speed.
Many drivers don't get that limitation of Tesla's system. Which is why Level 2 systems that get it right most of the time are dangerous. Tesla has a really good lane keeping system, which creates the illusion that it can drive well. Until it can't.
> Distracted driving is dangerous, claiming 3,142 lives in 2019.
https://www.nhtsa.gov/risky-driving/distracted-driving
> The National Safety Council reports that cell phone use while driving leads to 1.6 million crashes each year. Nearly 390,000 injuries occur each year from accidents caused by texting while driving. 1 out of every 4 car accidents in the United States is caused by texting and driving.
https://www.nsc.org/road-safety/safety-topics/distracted-dri...
Is it too much to expect technology to at least avoid the kind of accidents only incapacitated humans have?
"Pedestrians may enter the Interstate intentionally, often despite restrictions and controls, while drivers and motor vehicle occupants may become “unintended” pedestrians when their vehicle is disabled by a crash or other incident(Johnson, 1997)."
It could be argued that technically he wasn't driving.
I can't disagree. Autopilot in a car should mean the same as it does in "Airplane!" or "Wall-E".
> The driver of the Tesla, identified as a 22-year-old man from Lansing, was ticketed for failing to move over and driving with a suspended license.
And there's the actual problem. You can blame Tesla all you want, but you'll have a hard time proving that the brand/model/features of car this person chose to drove has anything to do with the crash.
The funny thing is that what an autopilot does in a real airplane is actually pretty close to what the Tesla autopilot does, if you set it to fly you into a mountain it will happily do so (abet with a lot of alarms going off in a modern airliner).
Sure, a basic private aircraft wing-leveler or heading keeping system will fly you into a mountain, but the autopilots in passenger airlines can take off, fly a complete course, and land the plane.
They're monitored by trained professionals with thousands of paid hours and hundreds of hours of training, of course.
CAT-III autoland is a thing but it's actually only used in very limited circumstances, not least because it increases runway safety margins in a way which restricts runway utilisation. Many busy international airfields don't even have a CAT-III certified runway. The aircraft won't automatically cycle through to the correct mode to land (there are a bunch of other steps required to put then autopilot into a double/triple redundant mode etc).
Shirley, you should keep your jokes PG! Wait, Airplane is PG? Oh man, the 80s were a different time...
An "Airplane! autopilot" requires... services... to operate. That was apparently PG in the 80s but not PG in today's society!