Judge orders trial in Tesla autopilot manslaughter case
courthousenews.com
courthousenews.com
I find the comment
"...we also know we are going to be sued...": https://youtu.be/VfyrQVhfGZc?t=3325
interesting grounds, for a lawyer request for internal Tesla emails via a legal discovery process.
To start with, Tesla assumes they will be able to accomplish self driving with the hardware they have. They also assume they need the system to drive on public roads (as opposed to simulations and private roads) to achieve this. They say that data collected from driving on public roads will get them there, even though this is a for-now-unsolved research problem in AI.
Furthermore, Musk seems to claim every year that Level 4 autonomous driving is within 6-18 months, going back to almost 10 years ago.
For now I feel there are too many people letting Tesla stick with their optimistic predictions and statements that they're "doing the right thing". Maybe the New York Times documentary that is about to come out will change this.
This is an absolute joke. There are videos of the autonomous "come pick me up" feature - whatever it's called - absolutely failing. One of them even runs into an airplane and pushes it around. Sometimes I think they're just using GPS with maps and PID loops and not actual AI of any sort. Nothing short of full general AI is going to be able to drive completely autonomously.
Yeah, I remember https://twitter.com/elonmusk/status/686279251293777920
> In ~2 years, summon should work anywhere connected by land & not blocked by borders, eg you're in LA and the car is in NY
Jan 10, 2016
They were full of this in 2016 https://www.tesla.com/blog/all-tesla-cars-being-produced-now....
> We are excited to announce that, as of today, all Tesla vehicles produced in our factory – including Model 3 – will have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver.
October 19, 2016
Mid 2019 he envisioned a million robotaxis by the end of 2020
Those did happen up until 2018
2014 - 90% capable of autopilot, for sure highway travel
2015 - autonomous driving for highways and relatively simple roads
2016 - S/X can drive autonomously with greater safety than a person - generally true, it still messes up for human-correctable situations but in general helps more than it hurts in terms of "crashes where autopilot was in control during impact or within 5 seconds of impact"[0]
2018 - incorrectly predicted - self driving will be 100-200% safer than a person by next year (this talking about their full self driving thing that will do turns and stop signs and whatnot)
the 2016 tweet about NY->SF was also incorrect by the time his prediction needed to be fulfilled, 2018.
I don't think they achieved "autonomous" driving in any way, since they require the human driver to be paying attention.
> 2016 - S/X can drive autonomously with greater safety than a person - generally true
Really? I think this is an extraordinary claim given some of the crashes and near-crashes that have happened (and again, the "autonomous" word).
Humans crash all the time.
It would be interesting to see a comparison of how often humans crash vs how often Teslas crash while the car is driving "autonomously".
I would not be at all surprised if humans fare worse, as there are plenty of horrible drivers, angry/distracted/inexperienced drivers, drunk drivers, sleepy drivers, etc...
Again, Tesla cars cannot drive autonomously by their own admission (they always require human attention), so this comparison isn't possible.
Even if that weren't the case, there's also the issue that people only enable Autopilot in cherry-picked situations, which don't statistically represent general driving situations.
Tesla instructs humans to be attentive while this feature is engaged, but obviously the car will keep driving even if the human's attention is elsewhere.
So Teslas do drive autonomously.. just not always as safely as (some) humans do under certain conditions.
Same with ordinary cruise control, which will keep the car going even when the human driver's attention is elsewhere, their foot is off the gas pedal and their hands are off the steering wheel.
While minor accidents are obviously much more common, in the US, there's approximately one fatality per 100 million miles driven. And that's across all drivers across all cars including old clunkers in poor mechanical condition and lacking modern safety systems.
On the other hand, I don't think I will believe the problem to be solved before I see a self-driving bike. I mean, I would propel it, while it would steer without my input (other than setting destination). Once I do, I will have near total confidence in robo-taxis.
The catch is that the situations where cars drive autonomously are easy, when the situation becomes complex, the autopilot bugs out and tells you to drive. Furthermore, the stereotypical easy case is on the highway, where you rack a lot of miles quickly, and statistics are typically "per mile driven".
Once you remove the bias, Tesla's self driving is not safer, I think I have seen a study that says that in the situations where Teslas can self-drive, they are on the same level as a human driver.
Maybe link to those studies. The one plus Tesla (and other ACC+lane keep systems that are obvious, as in 'loud beep when the system disengages'; my Honda notoriously gave the quietest beep whenever it lost its tracking of lane lines) has is that they don't look at their phone, don't get drowsy, and don't get distracted. So while many situations come up where a human could've prevented a crash, eg running into an overturned semi[0], it leads to about 3x less crashes compared to only passive safety features [1].
0: https://electrek.co/2020/06/01/tesla-model-3-crashing-truck-...
1:
> one crash for every 4.31 million miles driven in which drivers were using Autopilot technology (Autosteer and active safety features). For drivers who were not using Autopilot technology (no Autosteer and active safety features), we recorded one crash for every 1.59 million miles driven.
> we count any crash in which Autopilot was deactivated within 5 seconds before impact, and we count all crashes in which the incident alert indicated an airbag or other active restraint deployed.
Here: https://engrxiv.org/preprint/view/1973/3986
Discussed on HN here: https://news.ycombinator.com/item?id=30182141
(The one that didn't was the infamous Uber self-driving pedestrian fatality.)
Sure in 2022 - we (including Tesla engineers) are all comfortably able to say that Autopilot is really just fancy cruise-control and the driver really needed to have been using FSD (cuz that's full self driving). But at the time - if my memory serves correct Autopilot was being marketed as the feature state of FSD is today.
In the submitted article, it says a Tesla engineer testified that Autopilot was activated "about 20 minutes before the crash". I assume that means it was active 20 minutes before, and into the moment of impact. If this case does actually reach trial, I guess it'll be interesting howhat the jury will be instructed in weighing the use of Autopilot against the allegation of "gross negligence"
[0] https://apnews.com/article/tesla-autopilot-fatal-crash-charg...
Did you just invent a new word or have I been living under a rock?
>Riad had initially set his car’s speed to 77 mph; he then set it to 78 mph, and then finally changed the speed to 75 mph. At the time of the crash, the car was going approximately 74 mph.
For me, I’d convict. This isn’t a case of ‘autopilot’ as much as it’s a case where a person purposely (with intent) set his speed above legal limits and did not maintain the attention while doing so.. causing the death of two individuals on their first date.
Tragic.
(It turns out that's possible. California provides a GIS data digest of fatal accidents maintained by UC Berkeley. Data collection ends at 2021, but the 2016-2021 data set shows 22 crashes coming up on that intersection. https://tims.berkeley.edu/tools/gismap/).
Or we could limit the assignation of blame to the individual who is solely responsible for controlling his multi-ton high-speed vehicle at all times when operating in public thoroughfares.
... which is not to say they shouldn't do it. In fact, in the 21st century with ubiquitous communication we could even consider mandating that of all cruise controls. But if we were to codify that into law, we'd want a data source a touch more reliable than whatever Google, Apple, an Waze have cooked up as the speed limits on each road in the map (since those limits are currently at the safety threshold of "advisory to estimate travel times," not "someone could die if these numbers are wrong").
all the maps have built in speedometers with an alarming red rectangle when you are speeding.
the data for what the speed limit is is generally public, or very obviously derivable from principles-first, common sense engineering
autopilot has more than enough data to know its not on a freeway, and shouldn't be doing 60mph+ between nodes on a graph
By "all the maps," do you mean "Apple, Google, and Waze?" If so, see my previous statement; they're offering a driver-assist, but they're not ultimately responsible for your safety so their data doesn't need to be fully accurate. Tesla is pulling its data for the in-car map from Google. Nobody is going to try and sue Google if they're in a crash because "the little red box that tells me I'm speeding didn't show up, Your Honor." And Google has made clear that it indemnifies itself from such liability in provision of the map data (i.e. "You're holding it wrong" if you wire Google's idea of what speed limits are directly into the control plane a life-or-death machine).
So Tesla would have to take ownership of that problem directly... Put its own fleet of cars on the road to drive around and photograph speed limits. They could actually hypothetically do that given the cameras on the Teslas if they push data from the driver back to HQ. What's the data licensing agreement look like on owning a Tesla? Does it already grant Tesla that privilege by virtue of someone owning their car?
> the data for what the speed limit is is generally public, or very obviously derivable from principles-first, common sense engineering
Public, yes, but must be consolidated, validated, and kept up-to-date. And the principles-first "common sense engineering" does not describe how America's roads are actually labeled and structured, sadly.
> autopilot has more than enough data to know its not on a freeway
Does it? I don't think Autopilot has any concept of what road it's on; it's not consolidated with the Tesla's navigation map. It's viewing the road directly and looking for local lane markers and obstacles.
Such a consolidation with the global data to understand current speed limit would imply new design safety constraints for both the Autopilot and the onboard map systems.
It's also sometimes unclear about where transitions happen.
It also never accounts for actual speed of travel diverging substantially because citizens including Police have collectively decided that a different speed is more reasonable.
Basically such a system would probably be completely useless enough to ruin all cruise control.
do you actually drive in the US?
there's no shortage of stretches of perfectly straight road where the speed limit drops for no apparent reason.
Do you mean they're actually reading the signs as they pass on the road, or they have a map associating the current position on the road to a speed limit for that road?
If the former, the cutting edge of that technology I'm aware of isn't good enough to trust it to a safety system. And that's before we factor in added risks like "speed limit sign was obscured by a passing truck while the car drove past it so the car maintained speed."
If the latter, that's possible but it's not where Tesla is right now. Tesla doesn't own its maps, it's trusting Google, which doesn't certify the speed limits are correct enough in its map data to wire a safety system to that output.
I can't speak to what trucks do, but those are commercial vehicles, and have very different performance characteristics from cars.
That's an extremely strong assertion. Do you believe all car makers are breaking the law by making cars that can drive 80+ mph? By having cruise control that can be set at 80+mph?
In the US, that is an almost universally useless feature. Standard freeway speed is 10 over the limit.
It really depends. In my experience in the Northeast, when speed limits are 70-75 mph in more rural locations, I don't see most cars traveling at 85.
This is likely where we're headed... especially with fully autonomous vehicles.
It'll probably still be possible to hack them to go any speed you want, though... and certain cars like emergency and military vehicles will have no limit.
https://www.google.com/maps/@33.8729084,-118.2881048,3a,75y,...
I'm also extremely surprised that there's a 20 mph drop. While it's only a recommendation, the Uniform Vehicle Code and Model Traffic Ordinance suggests never changing the speed more than 10 mph at a time, and having no more than six such changes in a mile.
See page 28 of this PDF, as I was looking for this information specifically in relation to the Florida DOT, which does a pretty good job of following this rule:
https://safety.fhwa.dot.gov/speedmgt/ref_mats/fhwasa1304/res...
like a stop light?
The purpose of a traffic calming measure is to force slower driving via involuntary driver behavior or physics. Things like painting obstacle-like objects in the road to startle unattentive drivers, rumble strips, unnecessary curvature onto soft embankments so failure to make the turn doesn't risk injury, unnecessary narrowing of the road via cones or soft baffles (because people will subconsciously drive slower when they feel like their vehicle is inches from a barrier).
A common mis-design of highways is to assume traffic lights are sufficient for that purpose, and the result is creation of a high-crash location.
Has a traffic engineer ever been held legally responsible for accidents that result from their design?
In this sense the term “engineer” is someone that requires a license and training (like MDs or lawyers). States require engineers to “stamp” designs that they are taking responsibility for. Sometimes “city planners” LARP as (unlicensed) civil engineers and make designs that get built, which is somewhat problematic but happens.
Stamping and signing off on a design means legally accepting liability. How often they get prosecuted is rare. One famous example is Boston’s Big Dig fatal tunnel collapse. Of course politics and money had a big role in that case.
None. The answer to how much culpability the traffic engineer should have is "none". Freeways end that's what they do and there's no expectation that they won't have a light, or a traffic jam or a deer or some other obstacle in front of you that you'll have to stop for. Lights crossing two-lane highways happen all the time outside of urban areas.
Drivers are, appropriately supposed to be alert enough to see a red light up ahead. Of the twenty-two accidents you cited how many do you think would have been prevented with more signs?
For that matter it's not the autopilot's fault either its the fault of the idiot driving (or not driving as it were).
And when you dig into the five-year numbers, you find that while 22 is quite a bit, there have been more on the straightaway highways north-southbound east of the intersection and on the cloverleafs (If I were to hazard a wild-ass guess, based on California driving experience: driver going highway speed failing to realize traffic came to a standstill on the highway due to back-pressure of people exiting the cloverleaf at rush-hour).
IANAL, but exceeding the speed limit wouldn't be enough for a prosecutor to want to prove felony vehicular manslaughter. Even when DUI is involved, juries can still decide that the driver tragically f-cked up, e.g. underestimated how drunk they were, but not in a "grossly negligent" way. Even though virtually everyone knows that drunk driving is illegal.
But how many potential jury members know or believe that violating the fine print of using Autopilot is illegal or wrong? Or that a Tesla owner should be convicted of a felony because they "obviously" should have known better? I'd assume far fewer in a jury pool have strong knowledge and opinions on Autopilot versus drinking-while-driving.
I can imagine this being a major issue in this case. Elon isn’t known for keeping his mouth shut, especially when it comes to bragging about Tesla. I can imagine that his statements and Tesla marketing being compared to the fine print on Autopilot. Or worse, he tweets counter points live.
Especially complex if Tesla gets involved further as part of data discovery.
Is it understandable for you to set your traditional cruise control at 75mph and not expect things to change for 20+ minutes? I don't think so — I wouldn't trust CC to keep me on the road for more than 2 minutes without my direct guidance. Is it understandable that you might trust Autopilot to handle the road for 2o minutes, because you were confused or misled by Autopilot's capabilities? That feels like a substantively different question.
If you program a robot to commit an illegal act and someone dies, is it the robot manufacturer's fault?
It sounds to me like he was coming off the highway and didn't notice that he was leaving into an intersection with a red traffic light, where the accident occured. Not that I'm saying it is okay for him to not have noticed it.
> Riad was driving on the 91 freeway, heading west into Gardena. The freeway empties out into Artesia Boulevard at Vermont, but Riad, seemingly unaware of that transition, did not appear to react — and neither did his Tesla.
Surface street apparently. Maybe a 45/50/55mph zone.
(He was going 75 on a freeway; the freeway suddenly became a surface street, which is bad design, regardless of whether he should have been paying more attention.)
Not only that, but you can get a ticket for going too slow.
You also usually get tailgated if you're not driving over the speed limit -- just like everybody else.
One of the reasons I hardly ever use conventional cruise control is it makes you drive in a way that minimizes speed changes rather than doing what makes sense in the context of other traffic.
I’m guessing the criminal trial pleads out and the Autopilot issue will get more run in the civil complaints. I wouldn’t be surprised if Riad, the Tesla driver, files his own suit against Tesla to try to foist more culpability on to them.
> [Lawyer] argued that Riad’s actions — driving 74 mph on a surface street — did not rise to “gross negligence” and should not be considered a felony.
At that speed, the driver's momentum was around 67.000 kg*m/s.
That is an impact about 10 times bigger as getting thrown out of a plane and hitting the ground at terminal velocity.
I'd say it's gross negligence to drive that fast at all on any surface street with sidewalks, traffic lights, intersections, stores, whether the driver is asleep at the wheel or not, autopilot or not, drugged or not.
Agreed, but Riad set that speed when he was driving on a freeway, and Autopilot failed to slow down when the freeway became a surface street.
I think Riad was negligent for not maintaining control of the car (Tesla even warns drivers to "pay extra attention to the road" when using Autopilot or FSD), but I wonder if Tesla has any liability for marketing Autopilot and FSD capabilities that don't always work.
https://youtu.be/LEqM2FqUIa8?t=204
Surely the speed limit would change if there's a light-controlled intersection?
Every manufacture has some silly name for advanced cruise control. It’s called marketing.
Plainly false. Consider this guy, who was seen sleeping at the wheel three times, relying on the car to drive itself:
The thing that Tesla has named 'full self driving', but which it isn't is a completely separate product from Tesla.
But the point stands that it's bad and not something we should be structuring our society to encourage.
So? Any large truck moving below the speed limit and being perfectly safe can have vastly more energy. And a speeding motorcycle will have vastly less. In a highly-regulated behavior such as driving, negligence is not a function of energy. It is a matter of law rather than physics.
> That is an impact about 10 times bigger as getting thrown out of a plane and hitting the ground at terminal velocity.
That doesn't seem right.
Momentum is mass times velocity, p=mv. Average male weight is around 89kg, 74mph is about 33m/s. p_driver=33*89=2937 kg*m/s.
Terminal velocity of a skydiver is around 120mph, which is about 54m/s. p_skydiver=54*87=4698 kg*m/s.
p_skydiver/p_driver=4698/2937=1.6. So the momentum of the driver is smaller than that of falling at terminal velocity by a factor of 1.6.
Clearly GP's physics experiment makes little sense.
And why are we talking about momentum anyways? A fully loaded tractor trailer weighs about 80,000 lbs (~ 36,000 kg). Therefore, a tractor trailer traveling at 5 mph (~2 m/s) has a total momentum of 72,000 kg*m/s. By your argument, all tractor trailers driving through a city are grossly negligent.
What I wanted to illustrate was that those kind of speeds, in heavy vehicles, in cities, on surface streets, are negligent - spiritually. The sheer amount of damage that can be inflicted, in the blink of an eye, by a moment of inattention, is beyond measure (literally, that 67k number looks goofy, unreal, but it's cold hard physics). Our traffic laws are wrong, our streets are wrong. We don't tolerate that anywhere else. On a factory floor with dangerous machinery there are higher standards of safety than our public realm. This news article is the perfect illustration. Good chance the driver fell asleep, or was distracted in some way (totally normal human traits, btw, and does not make somebody a bad person). Yet the lawyer claims, that at 75mph in a 5000 pound car, this is basically just worth a speeding ticket. That's an enormous disconnect. That delta between responsibility and accountability is totally totally out of wack for drivers. An enormous amount of responsivblity (67k kg*m/s worth of it), and very little accountability ($100 speeding ticket).
And yes, at slow speeds, there's just a lot more slack in the system. The driver of a 5mph tractor trailer has a lot of time to react, so does every one around them, stopping distances are exponentially lower etc... It doesn't change the fact that it will level a house, and so we should have extremely high standards.
It is if you assume the car is a vintage box car from before the 80's where you might or might not have airbags at all. New cars are designed to crumble and deploy airbags to absorb that impact so that the squishy humans inside the car don't experience anything nearly that high. Nowadays the biggest risk is speeding, as it negates the advances in car safety.
> a 5 mph increase in the maximum state speed limit was associated with an 8 percent increase in fatality rates on interstates and freeways and a 3 percent increase on other roads
https://www.iihs.org/topics/speed#effects-of-speed-limits-on....
It's an argument against using some irrelevant momentum number in a comment. Without showing this momentum number, their comment is just "I'd say it's gross negligence to drive that fast at all on any surface street with sidewalks, traffic lights, intersections, stores, whether the driver is asleep at the wheel or not, autopilot or not, drugged or not" which certainly wouldn't get as much engagement on HN.
Even without computing any numbers, purely at the sanity-check level, this calculation is clearly wrong.
Think about it. While there have been some recorded incidents of people surviving a free-fall impact to the ground, that's mostly a certain death impact. Even if survived, you are in very bad shape (understatement). An impact 10x worse than that would completely destroy your body.
Then compare to race car drivers routinely having crashes at speeds higher than 74mph and most of the time hopping out of the car with no injury. Even at much much higher speeds.
The flaw in the argument is summarized in the common saying "It's not the speed that kills you it's the sudden stop". If you hit the ground from an airplane, the stop is very sudden, leading to extreme instantaneous g loads. That's what kills, not the absolute value of the speed itself.
In car crashes there is a lot of energy dissipation due to both structural crumple zones as well as motion of the car itself (it might spin, flip, skid more, etc). These factors lenghten the deceleration time and thus greatly reduce the g loads on the body.
I really hate these kinds of words.
A four lane "basically an interstate if it weren't for the traffic lights every couple miles" type boulevard or state highway typical of urban/suburban areas of sprawl type construction and a two lane residential street are both surface roads. And there's tons of both of those types of roads in the LA area (where this crash took place). There's a big difference between driving 74mph on a somewhat limited access boulevard vs driving 74mph on a residential street.
On a related note, I’m starting to think Tesla get a bad rap with all the negative press about their advanced cruise control system. And I didn’t call it ‘Autopilot’ here on purpose. Because there are many many versions of advanced cruise control on the market. Tesla does not have a monopoly on this type of technology. I saw an advertisement the other day for full size pickups that have it now.
The commercial showed a truck changing lanes and passing another vehicle, while towing a trailer, presumably all on its own. That is terrifying to me haha. Why do we all tend to focus on, and demonize Tesla’s version? Is it because it was one of the first, or maybe just the way Musk tend to be a magnet for criticism & scrutiny?
How many wrecks and deaths have been caused by other manufacturer’s systems? It’s got to be happening. This will continue to be a problem, not just for Tesla, as the tech becomes more widespread.
Because Tesla has been claiming fully-autonomous L5 self-driving “will be released next year” for half a decade now.
Not to mention robo-taxis, and the continued lies covering up their insufficient camera/perception hardware. Tesla needs HW5 at least.
FYI: Andrew Karpathy left Tesla a few months ago and is on garden leave. For better marketing, Elon calls it a sabbatical.
Unlike most others, Tesla has been very brazen to describe its system as self piloting rather than driving assists (but see the tons of asterisk at the bottom who clearly state that we're lying in the big letters!). Naming it Autopilot being one of the most obvious cases, autopilot means something and that's not what theirs does.
I know that things like EULA, NDA, and other forms of agreement like that are more easily enforceable there so maybe legally they won't be in trouble, but morally my personal belief is that they absolutely are.
The major difference is that Tesla pushes their system to operate in a wider variety of conditions, while still putting the same responsibility on the driver, and advertising the system using language that suggests that it is higher than level 2 functionality.
It should be illegal to advertise a SAE level 2 driver assistance feature as “self driving” (regardless of how much fine print you have); by definition, it never drives itself.
Tesla's system does more than the average level 2, so it's more useful. This may also mean operating it is more dangerous, but that isn't a given. Has it been demonstrated? If it has, I'd like to see that.
It seems to be mostly taken for granted in this sort of conversation, and it seems like if it were demonstrably the case, there would be a pithy blog on that subject online. I can only say I haven't seen that, but it could easily exist, this isn't something I focus on.
FSD is an egregious misnomer. It doesn’t take over driving responsibilities under any circumstance. “Full automation” under SAE J3016 is level 5.
The dangerous part is the false impressions of capability that they give to some of their drivers.
Advertising copy doesn't alter the car's safety, but it can make the driver of a car more or less dangerous. Tesla sells their cars as being capable of fully driving themselves, hands people a contract with a bunch of fine print saying "actually you still need to pay attention", and then acts shocked when drivers respond to the flashy marketing rather than to the fine print.
It's absurd that people still defend Tesla after all that marketing and place full blame on the drivers. Yes the drivers were careless, but Tesla intentionally created an environment in which drivers could believe their cars would take care of themselves!
[0]: https://www.kbb.com/car-news/mercedes-well-be-liable-for-sel...
Compare the marketing text for Tesla[0]:
> Tesla cars come standard with advanced hardware capable of providing Autopilot features, and full self-driving capabilities—through software updates designed to improve functionality over time.
to the text for Mercedes[1]:
> Driving assistance systems in S-Class: Intelligent Drive Next Level. With new and considerably extended driving assistance functions, the new S-Class will this autumn be taking a further, major step towards autonomous driving.
One of these is marketing responsibly in a way that makes it clear that the driver is still supposed to be in control, and it's not Tesla. In the marketing pages linked below, Mercedes uses the word "assist" 35 times and in 9 out of 14 headers. Read through Tesla's headers, in contrast, and you get "autopilot", "autosteer", "full self-driving". You come away with the impression that the car drives itself.
[0] https://www.tesla.com/autopilot
[1] https://www.mercedes-benz.com/en/innovation/autonomous/the-n...
My guess is that it's because Tesla themselves are calling the advanced cruise control system "Autopilot".
This risks presenting drivers with a level of misconception, and trust that this system does not deserve.
You can tell somebody (including Tesla owners themselves) plainly that this is no different than the other "advanced cruise control systems", that they should not put too much stock in relying on the system to drive themselves, and that they must be even more alert; however, I think there's a cognitive view in place -- even if you are told the contrary -- that causes people to think subconsciously that Tesla's cruise control is smarter. Almost like the car has a built in "Autopilot."
0: https://www.tesla.com/ownersmanual/model3/en_us/GUID-EDA7728...
1: https://www.tesla.com/ownersmanual/model3/en_us/GUID-69AEB32...
2: https://electrek.co/wp-content/uploads/sites/3/2017/02/autos...
I'd like to see others duplicate such tests and a response from Tesla on whether this behavior is as designed.
- If you go with the idea that Autopilot is dependable then it is Tesla's fault. The system is supposed to prevent that from happening and it didn't, killing two.
- If you go with the idea that Autopilot is just an assistance then it is the driver's fault, but it also means that the system doesn't bring much value. If you have to pay attention at all time, correcting mistakes, then you may as well just drive.
the system is very clear that you should be monitoring the car while driving, all the time.
>If you go with the idea that Autopilot is just an assistance then it is the driver's fault, but it also means that the system doesn't bring much value. If you have to pay attention at all time, correcting mistakes, then you may as well just drive.
Value is a subjective proposition and not inherent to anything but your own judgement.
We actually do have the numbers on this, because other manufacturers actually report this data.
Tesla has more self-driving accidents, and more fatalities, than the entire rest of the auto industry, worldwide, combined.
This will continue to be a problem, not just for Tesla, as the tech becomes more widespread.
This is uniquely a problem for Tesla, not the industry, because the rest of the industry took safety issues into account when designing their systems and testing them. Tesla deliberately did not for profit purposes.
While I don’t think the driver is without serious fault here, shouldn’t Tesla really be able to slow and stop a car coming off of a freeway and presented with a controlled intersection?
Don’t get me wrong. I own a Tesla and use Autopilot. I also think that the combination of map and sensor data should clearly prevent incidents such as this one.
Maps will always lag reality; I don’t think that it’s reasonable to expect Tesla maps to have perfect accuracy.
The bottom line is, the car is NOT self-driving. If you’re behind the wheel, you’re driving and you’re responsible, just like any other car.
Maps will always lag reality, but this is a commonly driven street, probably not the first Tesla to drive down it-- Waze was using info like this for years, hate to think Tesla is less capable.
In short, there is no excuse for Tesla's maps not knowing about the speed change on this stretch of road.
With FSD beta, turning off a major highway onto my neighborhood street (very narrow and no lane markings), it defaults to 25 MPH. About 100 feet later, the car sees the 35 MPH sign and speeds up accordingly. In this case, 35 MPH is too fast given the curves, slight hills, and limited visibility. I slow down manually in these cases using the roller wheels.
Tesla has gradually improved its speed planning to account for different scenarios. But it's still a work in progress and often requires manual tweaking.
That's why you don't rely on maps as your only input to a feature such as this. Also, is there any information on how long this situation was in the way posted because I can't imagine that they built a city overnight at the end of that transition, it looks like a situation that had existed for many years prior to the accident.
An autopilot in an airplane does not absolve the pilot of responsibility to maintain vigilance and avoid unsafe conditions. Same in the car.
This is false, and I don't know why Tesla stans keep bringing this BS up like it's some sort of justification.
A modern aircraft autopilot is fully capable of taking a plane from takeoff to landing without any pilot intervention required. Pilots frequently use autopilot in flight, but rarely do so for takeoff or landing because it's more stressful to monitor the craft's autopilot system at those moments than it is to simply handle those parts of the flight themselves (and at least for autoland functions, the airport must be certified for autolanding).
There are also tens of thousands of Cessna’s etc. that have the functionality I described and no more (sometimes less).
There's lots of things the media has called Autopilot.
The media, or Tesla? It's not the media's fault if there's confusion on the several marketing terms Tesla has come up with to imply it will drive for you.
Cars are inherently unsafe when the driver is drunk, would you say cars are either safe or shouldn't exist and be illegal?
There's signs for miles back. The speed limit progressively slows down to surface street speeds. If a Tesla can't prepare for a freeway ending then it shouldn't be on the road.
No.
But other car manufacturers are not claiming to “drive you from LA to NYC with zero interventions”
https://www.theverge.com/2021/5/7/22424592/tesla-elon-musk-a...
https://www.tesla.com/autopilot
Autopilot Full Self-Driving Hardware (Neighborhood Short) https://vimeo.com/192179726
i saw this:
“Current Autopilot features require active driver supervision and do not make the vehicle autonomous.”
"The person in the driver's seat is only there for legal reasons. He is not doing anything."
I agree - though - as someone who holds Tesla stock indirectly through various indexes - I fully can understand the desire to want post facto look well past the webpage fold for fine print.
Can my neighbor below sue my washer manufacturer if they got flooded?
That’s the billion dollar question that will be relitigated for years as autonomous systems improve. No one knows.
Edit: I thought this is in Germany, so I'm wrong. My bad.
In civil law jurisdictions, this works differently.
https://www.nytimes.com/2022/02/15/nyregion/sandy-hook-famil...
The NYT describes the plaintiff's argument as focused on Remington's marketing strategy:
> The families contended that Remington violated state law by promoting the weapon with an approach that appealed to so-called couch commandos and troubled young men like the gunman who committed the Sandy Hook massacre.
> Nicole Hockley, whose 6-year-old son Dylan was killed, said the documents included in the settlement were crucial — and “paint a picture of a company that lost its way, choosing more aggressive marketing campaigns for profit.”
To sue someone, you need to have incurred damages of some sort and file a complaint alleging that who you're suing is responsible. At a high level, you can sue anyone for anything.
Legally the driver bears the responsibility, no?
This is a really vague question. Bears the responsibility for what, exactly? Are they solely responsible? These are things that exactly get sorted out in court...
Can my neighbor below sue my washer manufacturer if they got flooded?
Yes.
This is false advertising, and apparently it may be involved in turning Tesla drivers who believe it into negligent killers. The driver may be able to argue that Tesla told him the car would stop, and so there is no negligence on his part.
However, we've made getting licensed to operate a vehicle way too easy. We should also make drivers get training on equipment type so switching from a mid-sized sedan to a gianormous SUV requires more than just being handed new keys.
The extent of training increases the investment a person has around the regulated activity. The person is more likely to abide by the rules because (a) they have a deeper knowledge, and (b) they don’t want to lose the privilege of being able to carry it out.
Increasing the strictness of license requirements could be a small nudge towards decarbonization to slow the growth of personal vehicles on the road.
Sadly, I'd have to disagree. I think all this would do would be to increase the number of drivers operating illegally. Increase the friction, and people will just find a way to avoid it all together. The only real way to stop it would to have mandatory checkpoints. The push back from the masses for any kind of change like this would be enormous that you'd be asking Roe v Who? in comparison.
But honestly I think demands to make a driver's license harder to get largely miss the point. From my own experience, time on the road makes far more difference than training or harder tests. As does, general maturity and attitude which isn't going to be taught by a class.
What does this have to do with anything? Advocating that people that can't afford to take training shouldn't be restricted from getting licensed? With that logic, let's all become pilots for free too!
>time on the road
The rest of this I can almost agree with, but you're glossing over that bad habits become entreanched. Distracted driving from today's smartphone use to the classic putting on makeup while driving, constant changing of the radio/spotify/etc, or just shoving fries/chips down your gullet while driving is never going away no matter how many years you've drive. In fact, the longer one goes with out having a wreck with these bad habits, the more one is emboldened that they are safe.
It's not just teenagers with low hours behind the wheel that do any of the things listed above.
So, you must be the change you wish to see in the world. So in today's view, we're all screwed!!
I think we're in general agreement. My original point was that making getting a driver's license a leetcode interview-level requirement puts more barriers in place but probably doesn't (by itself) mean you have a lot better drivers on the roads.
To be clear, if you can afford it, a high-end driving course is probably both useful and a lot of fun. Though I wonder if it's more useful for someone who has some experience. (I've never take a course though I have done some driving on BMW's track in SC which was a great experience.)
There was a post here a while back where someone built a keyboard that would deliver shocks when it detected improper typing techniques. It wasn't enough to do anything other than make you really want to type well. It was funny that they went through with it, but that's the level of schtick I'm intending by gentle shock.
It is impossible to get insurance, plates, tabs, park your car in your apartments lot without proper licensing.
Everyone eventually messes up and gets pulled over or gets in an accident and then you are basically going to jail.
In theory your cousin can buy and register your car but if you kill someone he will be legally and monetarily responsible.
Logically it would have to be fantastically burdensome to convince many people to drive without licenses.
It is silly to both pretend we absolutely must impliment a law either without the slightest bit of sense or not at all and presuppose that everyone is going to ignore the law when clearly this is actually fairly hard to do.
Based on experience (admittedly biased), plenty of people are driving without a valid license or insurance right now.
That’s not an argument not to increase proficiency standards, but I think people who naturally follow rules (like my spouse) have a hard time comprehending people who would see a rule and then evaluate whether or not to follow it.
If you are operating a car in someone else's name does not mean they are ultimately on the hook. Yes, they will probably be main suspect, but they will/could be able to prove not involved. Easiest would be to say you were using the car without permission.
>Logically it would have to be fantastically burdensome to convince many people to drive without licenses.
WAT! this happens every. single. day.
*I know direct 2nd hand is counter-intuitive, but I mean I personally know people with cars like this.
In Germany, a court ruled that they misled customers on the capabilities of autopilot and banned advertising it: https://www.cnbc.com/2020/07/14/tesla-autopilot-self-driving...
And it is "statistically" which is how all safety measures are evaluated.
It turns out it's a lot easier to not crash on a highway than to do it on surface roads with many stops and starts.
https://www.tesla.com/autopilot
And note that this is the current marketing material.
Edit: It's not mentioned in the article, as far as I see, but it's likely the driver was on the cell phone. It mentions the data shows the driver had their hands on the steering wheel, but that can be easily thwarted, to my understanding. The fact that they didn't brake at all implies they weren't actually holding the steering wheel, at least in a way where they were paying attention. In my opinion, both the driver and Tesla bear responsibility.
Current Autopilot features require active driver supervision and do not make the vehicle autonomous.> Please note that Self-Driving functionality is dependent upon extensive software validation and regulatory approval, which may vary widely by jurisdiction.
And the rest is marketing copy explicitly written to make it sound more fully-featured than it actually is.
[1]: https://web.archive.org/web/20180211121851/https://www.tesla...
> It is not possible to know exactly when each element of the functionality described above will be available, as this is highly dependent on local regulatory approval.
So while you can argue that the marketing is deceptive, that can't really be read as saying that you could let the car drive itself at that time.
Well I suppose it'll be up to the judge to determine if a relatively recently added disclaimer on the website trumps years worth of bold claims and tech demonstrations.
Tesla can’t have it both ways. Tell everyone that they have autopilot/self driving to sell car and build hype, but then claim that the website says the opposite so there’s no problem.
/s
Autopilot and FSD are two different things ... I'm sure the consumer is fully aware of that /s
The car doesn't ship with autopilot or FSD turned on.
The general public is extremely stupid and believe everything they see in movies/TV. In fact a large part of the population believe that Fentanyl can kill you just by touching it which is the most ridiculous thing ever. Yet people are so paranoid about it that police officers have collapsed in hysteria after touching Fentanyl. [1]
Candy wrappers say "Do no eat!" on them in the Unites States so you can not assume people know how an actual Autopilot works. Now you may say this is because companies are getting sued but that proves the point that you can actually sue someone for being this stupid and win.
[1] https://healthandjusticejournal.biomedcentral.com/articles/1...
Bendix/King autopilot has a feature called Enhanced Ground Proximity Warning System which would be hard to ignore. Some models have "terrain avoidance maneuvering" which would actually prevent it.
“Terrain, Terrain, PULL UP, PULL UP”
Hopefully you were not sleeping. ;)
On a related note, although they usually mean something's going wrong, which isn't cool, I've always been impressed by the way those warnings are delivered in airplanes. The tone and cadence of the warnings really communicate that this is something serious and you should respond to it and also communicates what exactly you should do.
Here's a video of the exact scenario being discussed in a Boeing 747-400:
Fighters with FBW like the raptor can't be over G'd because FBW takes precedence over pilot input.
That said, the selling of FSD has, to my knowledge, always been positioned as the selling of hardware that is expected to be capable in the future of FSD. Sometimes you have to read all of the words in a paragraph to get that meaning, but that is the clear reading I get from that same autopilot page:
Full Self-Driving
All new Tesla cars have the hardware needed in the future for full self-driving in almost all circumstances. The system is designed to be able to conduct short and long distance trips with no action required by the person in the driver’s seat.
The future use of these features without supervision is dependent on achieving reliability far in excess of human drivers as demonstrated by billions of miles of experience, as well as regulatory approval, which may take longer in some jurisdictions. As these self-driving capabilities are introduced, your car will be continuously upgraded through over-the-air software updates.By the same logic an Hello World Python script has the capability for general AI, i just need to wait enough years then add later add the missing part that accidental is the essential thing.
Nobody disputes the terms of service language, but the advertising language, the intentional misleading names and stats greedy companies use. We are sure lawyers checked those terms of service very well and worked fine for US, seems German judges are not tricked by this trickery though, would be nice if more countries punish missleading advertising.
I imagining new Elon product "Clean Drinking Water" and ads showing people drinking it, BUT with small text that mentions that the "safe for drinking will happen in 10 years after we fix our shit" and fanboys defending the naming because you should already know that Elon is always doing this and you should read ignore the advertising and go to the legal documents about the product to figureut all the real details.
I don't fly but have an autopilot on a boat, same thing. It'll happily hold a heading onto the rocks if I set it. But everyone knows that's what it does, the company doesn't make any claims or insinuations otherwise.
How is it advertised?
The marketing is weird. They mix up current capabilities and future aspirations deliberately.
For instance, here:
> The system is designed to be able to conduct short and long distance trips with no action required by the person in the driver’s seat.
They actually mean that the system is designed such that "in the future" it will be able to do this. The previous sentence is "All new Tesla cars have the hardware needed in the future for full self-driving in almost all circumstances", and you're supposed to carry that context forward into the following sentence.
Another aspiration, which if you weren't reading carefully you might assume was a present capability:
> All you will need to do is get in and tell your car where to go. If you don’t say anything, your car will look at your calendar and take you there as the assumed destination. Your Tesla will figure out the optimal route, navigating urban streets, complex intersections and freeways.
It's intentionally misleading.
https://www.tesla.com/en_GB/autopilot
---
PS. Yes, there are caveats: "Current Autopilot features require active driver supervision and do not make the vehicle autonomous". That doesn't make the rest of the marketing copy clear.
https://mobile.twitter.com/elonmusk/status/79991021385159065...
"Full Self-Driving Hardware on All Cars
All Tesla vehicles produced in our factory, including Model 3, have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver."
Now, I can see how in a court of law you can make various arguments about what that promises, but I can see how someone would interpret that as it can drive better than they do.
There are plenty of laws. Most products come with more pages of safety warnings than are in the rest of the user's manual. And, in California, you're informed that pretty much every building you enter or product you use causes cancer.
That it's far safer than the average human driver?
If anyone would read that as the car driving autonomy being far safer we would have fatal crashes every day.
This is in direct contradiction to Google/Waymo's research on the matter.
Does Tesla have research to back up this claim? Of course they do not.
Musk is in Brazil as we speak claiming the cars will be driverless within the year.
"An autopilot is a system used to control the path of an aircraft, marine craft or spacecraft without requiring constant manual control by a human operator. Autopilots do not replace human operators. Instead, the autopilot assists the operator's control of the vehicle, allowing the operator to focus on broader aspects of operations (for example, monitoring the trajectory, weather and on-board systems)."
I don’t like this argument, on the grounds that the average driver has no reason to know what actual autopilots do — I certainly don’t — and therefore naturally go by what it sounds like it does.
Even a cheaper autopilot in a Cessna can take you from the airspace above your starting airport to the airspace of your destination, including lining up final approach maneuvers, with zero input from the pilot. Modern planes absolutely can fly and navigate themselves, but for safety, operational, and policy reasons pilots often don't use the full capability of these autopilots.
From Tesla:
"Full Self-Driving Hardware on All Cars"
followed by:
"All Tesla vehicles produced in our factory, including Model 3, have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver."
Self-driving cars doesn't sound like fancy name for cruise control.
That's from Tesla. Who has already faced consequences for lying in marketing.
Even if they weren't trying to mislead people in their marketing materials, these points seem enough to make a case.
What are you talking about? When was the last time you saw a Tesla advertisement?
> Self-driving cars doesn't sound like fancy name for cruise control
Having the hardware or the capacity to be a self-driving car is not equal to being a self-driving car. The hardware must be enabled, which in this case wasn't. The car only had Autopilot not FSD. Please learn the difference.
People say Teslas are safer than humans, but humans who can't see large broad stationary objects would never be granted a license. If you are a robot driver, no problem though apparently. In fact, if you have proven to be recklessly homicidal while driving, your license is taken away and you are likely put on trial. Not if you're a robot! They will clone your flawed sensors and robot brain, and ship it to thousands of people to be deployed on roads across the world.
When your web service goes down you probably find the root cause, fix it, and issue a report about how it's not going to happen again due to your fix. Not Tesla! They are aware of the problem, know exactly how to fix it, and it seems like one man's ego is getting in the way of shipping actually safe autonomous cars. Tesla cars are not safe because Tesla as a company does not value safety as a priority. They value design, whizbang technologies, branding and image, while safety is a distant concern. Elon Musk put more effort into naming the line of cars SEXY than he did to actually producing reliably and predictably safe autonomous vehicles. Musk spent far more time lamenting over the fact he was forced to name his car the Model 3 than the fact it decapitated a human being due to his poor engineering choices.
Teslas on AP are death machines (read: they are machines who have killed people) and we in the public are the unwitting beta testers for Tesla Inc.
I'm with you on this. Nobody deserves this and this software should be in a test lab or closed road course until it's done. And others will argue that you can't perfect automated driving in those situations, and that's correct.
Yeah I can't help but cringe at this. Safety is hard, and you don't get safe systems without trying very hard and deliberately to make them. That's a fact of life and engineering. Trying hard to make safe systems does not look like deploying 2 ton automobiles that are critically blind in the very arena in which they are expected to operate. That's called reckless endangerment in my book.
Which makes it all the more galling and reprehensible.
> makes it clear that your concern isn't really with the engineering limitations and safety implications of this product.
I was concerned about engineering in 2015 when the first decapitation happened. My present screed against Tesla is because they haven't fixed the problem in the intervening 7 years to the point where a second person was decapitated. That shows me the problem is not engineering but organizational. It makes no sense to talk about engineering issues when Tesla does not promote a culture of safety. Engineers are not in control there, hype men are.
The number of accidental decapitations your product gets to make is 1 and that's arguable. The second decapitation is criminal negligence.
And yes, I am incensed by this because as a roboticist, I think it's a travesty that Tesla has taken the industry in this direction.
Imagine if it were your loved one who died in a Tesla using a technology that Tesla knew or should have known to be deficient and intentionally oversold its capabilities. Would you shrug and point to industry statistics? Or would you be mad at Tesla? Be honest.
Frankly, the attitude that Tesla should be allowed to test their beta-quality products on public roads (I certainly didn't agree to participate in the beta test) is what I find to be ridiculous.
But isn't one of AP's selling points is that it's sophisticated enough to mitigate if not outright prevent these kind of crashes? In previous fatal AP-involved accidents, Tesla drivers collided with stationary objects, which apparently are particularly difficult for AP to see. This case seems to involve one of the more common kind of serious head-on collisions: failing to yield to left-turning vehicle which has the right-of-way.
I know AP != FSD, and that AP doesn't necessarily promise it can handle traffic lights. But if it can't detect or react to vehicles slowly moving into the driver's lane...What are the dangerous day-to-day situations that it can confidently and reliably handle? Is it basically just lane control?
That person being the worst driver in the world.
Watch the first video there and how it begins.
Its unclear which autopilot features the driver thought he had, but if believed he had purchased the "Enhanced autopilot" and that it was enabled wouldnt that mean the Tesla's system failed here? The intersection in question is at the end of a freeway[1] which would constitute an exit.
I do agree with most that the driver is partly to blame, but probably not as a felony manslaughter. Tesla on the other hand should be held criminally responsible since at best they are misleading customers into believing their car has features it doesn't and at worst they are claiming a level of system reliability in their product that just does not exist (something I would classify as gross negligence on their part).
My father has one of these cars with the enhanced self-driving enabled, and I am always gripped in mild terror whenever I ride with him, regardless of whether it is the car or my father who is driving...
[0] https://www.tesla.com/en_AE/support/autopilot-and-full-self-... [1]https://www.google.com/maps/@33.8715442,-118.2917755,17.35z
The case seems pretty clear cut anyway, the guy had cruise control on and wasn't paying attention to traffic lights. We'll see if he turns around and sues Tesla, pleading ignorance about what "autopilot" could do.
Just like we were in 2014: https://www.theverge.com/2014/10/2/6894875/elon-musk-says-ne...
> . A Tesla car next year will probably be 90 percent capable of autopilot. Like, so 90 percent of your miles can be on auto. For sure highway travel.
I don't see a reference to "full self-driving" and certainly not something that'd stop for stoplights and properly handle traffic signals.
In the past, you could order the cars with or without FSD hardware (on promises of future capability). These days (~2020+? I'm not sure the cutoff), you can order them without or without an FSD hardware enablement/cost, but they all have FSD hardware inside them, and the cost is just about turning it on or not, and you can always later opt to pay the cost or a subscription to enable it.
For cars that do have FSD hardware both installed + enabled (paid-for): if you're part of a smaller select group of opt-in, approved testers (currently ~100K drivers in the US plus a smaller number in Canada), you can enable "FSD Beta", which is a whole different animal than Autopilot and not really relevant to the discussion.
However, when you're using regular Autopilot, the capabilities of Autopilot are different for cars which have installed+enabled FSD hardware, and those which do not (either because you have the hardware but didn't pay the enabling cost, or because your car is old enough that some of them didn't come with the hardware at all if you didn't pay the cost up front or pay to have it retrofitted).
Non-FSD-capability cars, for example, can't detect stoplights on Autopilot at present, while FSD-enabled cars running Autopilot can. There are probably some other differentiators like that as well. Basically, Autopilot-on-FSD-capable-cars is slowly getting minor Autopilot feature upgrades that are only possible with that hardware while they're waiting for the "real" FSD to arrive. Autopilot-on-non-FSD-capable-cars doesn't get these upgrades and is stuck with a more-limited functionality.
The best argument Tesla has to isolate itself from liability is that there is no way that the car could have reacted to the oncoming car and stopped at that speed, even if its AI system were perfect. They would have to dodge criticism about not using LIDAR to detect obstacles early and also the criticism that the AI system didn't notice the obstruction _at all_ and could have slowed down earlier.
All that said, this is not a good fact pattern for the people hoping to come after Tesla instead of the driver. However, the cracks are beginning to show.
There is never a legitimate was to be asleep at the wheel. He was perhaps genuinely asleep, but short of being drugged against your will there is no non-negligent reason to be asleep at the wheel.
Edit: to clarify, the adverb "legitimately" modifies the verb "have been" in this context. You can tell because it is immediately before "have been." If I had said "he was legitimately asleep" that would mean that there was legitimate reason for him to be asleep, because the adverb "legitimately" was attached to the word "asleep." The phrase "he legitimately is asleep" says nothing about the legitimacy of the sleep, but emphasizes the legitimacy of the "is" (as in, he was so asleep that he wouldn't notice his car beeping at him to wake him up).
https://en.wikipedia.org/wiki/R_v_Parks https://en.wikipedia.org/wiki/Sleepwalking#Homicidal_sleepwa...
Then they shouldn't have been driving at that speed.
1. The traffic lane the victims were turning into is in addition to the ending freeway's traffic lanes (i.e. it is one lane further to the right than the right-most through lane). So someone had to be crossing lanes one way or the other for the collision to occur in the first place. Maybe not, I am not a traffic expert or familiar with the area, but that is how it looks to me as an experienced driver.
2. If "Autopilot" was engaged why didn't the Tesla react to the obstruction? Perhaps the timing was too sudden and it didn't have time to reach before being smashed to pieces?
3. Was the victim driver tested for intoxication? Perhaps they were not following all traffic laws as well? Sometimes accidents are the culmination of multiple drivers' mistakes. Not trying to victim blame though. May they RIP.
I think it is clear the Tesla driver was being reckless here because of the speeding. It is unclear if he ran the red light or not. I'd say at the very least revoke the guy's license permanently.
Tesla is, to me, clearly reckless as well with their marketing tactics and their failure to tamp down unrealistic expectations and the hype-machine surrounding "FSD".
0: https://www.usatoday.com/videos/news/have-you-seen/2022/04/2...
The victims were making a left turn to get to the other side of the road to head east, the defendant was heading west.
> Perhaps they were not following all traffic laws as well?
They were the fourth car turning left. If the light was extremely short, maybe the light had turned red and they were late, in which case they were in the intersection illegally. NB that were that the case, they still would have right of way as they had entered the intersection before the Tesla. The defendant would still be fully culpable.
I don't see any situation in which the defendant was not driving recklessly. He was almost certainly driving at 74 mph directly at a red light, unless all four cars ran a red light across multiple lanes. When the light turned green he had at most a second or two to have stopped, if it had not changed. He did not even touch the brake.
Realistically, he was driving with his eyes closed because it was 1230 and he was tired. Emergency braking would not have saved the two people he killed. Evasive driving would have still caused a crash at that speed. The only thing that could have saved their lives was respecting the red light, and the car was not programmed to do that.
Even if the car was fully self-driving, the "hands on steering wheel" thing is a very clear "pay attention" signal. There's a valid argument that complacency is a natural result regardless, but this was not complacency. He had zero awareness of what was happening outside the vehicle, and that is reckless.
I've been saying this (and getting downvoted) for years: AI is a tool. It is no different from any other tool in that the person using it is responsible for whatever it does. If something happens when an AI is involved, it should be presumed to be at fault. And thus the user of the AI is also presumed to be at fault.
This comes into play specifically with the "child in the middle of the road" scenario. At some point our vehicles will become good enough to make actual life and death decisions. Does it swerve into a tree, into oncoming traffic, or take out the child? The only moral, ethical and practical answer is: An AI should choose to harm its user rather than anyone else, because there can never be 100% certainty that the AI did not make a mistake.
You choose to use Tesla autopilot and it kills someone? You are responsible. Period. Feel free to sue Tesla for selling you a faulty tool, but you're still culpable for it's use.
How it's going: People died.
I found this: https://www.autopilotreview.com/cars-with-autopilot-self-dri...
But the site is very Tesla-oriented, and I’m not sure if the info is up to date for other automakers. Also not sure who the publisher is or whether they are credible.
I’ve personally tried out Tesla, Toyota, and VW and the Tesla was definitely the best at the task of driving on the highway in traffic and staying in the lane when the road turned.
IMHO, computers and software cannot, yet, and for a long time, will not, account for all the variables that are part of driving. People that drive these cars really need to understand that computers cannot think like people. Thats my two cents about this, and I hope that this trial puts the brakes on testing this tech in public.
Neither can every person who's allowed to drive.
The question is should we allow semiautonomous/autonomous vehicles that drive at least as good as the worst human drivers who are legally allowed to drive?
If not, why not?
Likely almost everyone agrees a excellent-human-driver general-AI grade of full automation should be legal; with a to-be-determined degree of manufacturer liability when a collision occurs.
All of the dispute between those extremes boils down to this:
Should there be a range of automation that is illegal to make and/or use? What should that range be? What current cars fall in the range? Should the illegality be based only on the technical performance, or also on the branding? When such illegal use happens, who all should be liable / punished?
Riad hit and killed the couple after a left turn from exiting the freeway.
So either speed limit data was missing (in which case the limitation doesn't apply), he had his foot on the pedal and didn't know it (during which Autopilot will not brake; the driver is notified of this if the pedal is pressed down for more than a few seconds), or he turned Autopilot off before the crash (unlikely given the data that's in this article).
I don't remember such a limit on my M3 but will check tomorrow. You can simply accelerate to any speed below AP's global limit (90mph in this case) and then engage AP. It'll keep at that speed. Meanwhile, Model S'es manual doesn't seem to mention anything in this regard.
[1]: https://www.tesla.com/ownersmanual/models/en_us/GUID-5033143...
This is false.
But I guess it is a top down culture issue perhaps?
Seeing how Elon Musk acts, and how he will chase the shiny thing when he wants attention, it’s all about ego right? Like what’s with the SpaceX always on demand massages for their executives?
Seems like they are losing sight of what really matters, and I can’t help but wonder if this build up of failure is going to cascade into the Musk companies faltering, which would be a big loss as space flight and green transportation are very important (imo)
SpaceX has had onsite massages for a very long time as a comfort perk to all employees -- no different than FAANG perks https://www.protocol.com/bulletins/google-massages-return-to...
> losing sight of what really matters
I wouldn't say so. If anything his companies are doing better than ever. Tesla is doubling revenue / deliveries YoY with insane gross margins and just opened 2 new factories on 2 continents. SpaceX is sending a new batch of Starlink satellites is going up literally every week, and they've sent over 20 people to LEO and back (could you imagine if we had to rely on Russia for this today?!).
Not to mention that the autopilot program is full of a ton of really smart & hard working people who put their soul into their work, because the lives saved by bringing the product to fruition will be astronomical. They are making real practical progress on AI systems -- better than can be said of much of academia & industry.
That's an insane speed to go somewhere that isn't decisively apparent to be a highway.
not an insignificant detail. sounds like the driver was a menace.
Tl;dr: 45mph or less
I used to think the same, but after some years, I came to the realisation that Tesla is just the Silicon Valley's idea of a car company. They have been releasing unfinished products with glaring quality defects for a while now, and they have overpromised almost since the very beginning.
I honestly can't wait for more car makers to go fully electric, so consumers have better choices.
So, yes, over the coming years I expect we'll see much more innovation in this area from established manufacturers (or even other new ones). But not sure how Tesla itself will play out.
You assume they won't try and copy the Tesla product development cycle as well.
I think Ford, GM, VW, etc. have been on the end of enough accidents, recalls, and lawsuits to understand the dangers and consequences of what they’re doing and have their own safety standards even if all government standards went away overnight.
So I don’t think it will happen for them.
Tesla is not operating that way. I don’t see how they don’t end up at a reckoning.
This probably isn’t it. But it seems like it’s coming.
When some of us warned of this shit from the start we were called luddites, impeding progress, boomers afraid of the future.
I still want my promised hoverboard, instead we get an upgraded cruise control DLC laden car that kills people while claiming you can watch movies and sleep while it drives you.
It is about time that this feature is being under investigation.
In those clips you see the fsd computer avoiding accidents form the sides and the back, cases where we humans cannot see before it’s too late.
It’s not perfect yet, but it will be superhuman in term of safety in the near future.
That is exactly what this case (a routine felony manslaughter trial) is not doing though.