Tesla braces for its first trial involving Autopilot fatality
reuters.com
reuters.com
The excuse is that this firmware version is not available to the public. Yet Elon is driving in public with other drivers putting everyone at risk of death. At what point are we going to stop this lunacy?
That video does nothing to change that. Why should it?
https://www.tesla.com/VehicleSafetyReport
https://www.teslarati.com/tesla-fsd-beta-safety-crash-statis...
Both autopilot (basically lane-keeping) and FSD have multiple times higher miles driven between crashes than the US average.
As per the company, a Tesla with FSD Beta engaged experienced an airbag-deployed crash about every 3.2 million miles in the last 12 months. This makes the system about five times safer than the most recently available US average of 600,000 miles per police-reported crash. It should be noted that FSD Beta specifically works for inner-city streets, which involve tons of edge cases and unpredictable driving behaviors from other vehicles.
I really find it hard to understand how anyone who has driven a Tesla with AP for any distance thinks it is safer than a human driver. Maybe just drivers who are always texting or drunk. AP is the least defensive driver on the road, it needs close supervision.
Do you trust VW if they first party report their emissions? Philip Morris if they first party report the effects of smoking on lung cancer? Unaudited first party reports are literally worthless. They have every incentive to lie or misrepresent.
For that matter, Tesla has repeatedly, actively lied to consumers and misrepresented their products.
The current director of Autopilot software, Ashok Elluswamy, admitted under oath to staging the initial “Paint It Black” Autopilot demo video claiming full self driving capability: “The person in the driver’s seat is only there for legal reasons. He is not doing anything. The car is driving itself.”.
The company misrepresented the range of their cars and actively suppressed complaints and gaslighted customers requesting service [1].
As for the “safety statistics” themselves, they are completely devoid of any information. The only information reported is a single ratio munging both Autopilot and FSD together.
They do not break out the products. They do not enumerate the crashes. They do not even report the number of crashes or number of miles used for calculating their “statistics”. They can not even be bothered to publish the damn numerator or denominator of their safety numbers. This is grade school level reporting and they are being allowed to drive 2 ton machines on public roads.
In contrast, this is what Waymo reports: https://storage.googleapis.com/waymo-uploads/files/documents...
A 30 page research paper itemizing the specific crashes. Methodology, references, evaluation models, comparisons, etc. That is what a proper safety report looks like, not the grade school bullshit Tesla marketing is pushing. Tesla safety and compliance should be ashamed of themselves for prioritizing marketing over human lives.
[1] https://www.reuters.com/investigates/special-report/tesla-ba...
This is obviously worse because you have to be paying attention and be able to quickly take over, but it's not like this type of thing would never happen with a human driver.
With their software, Tesla is more of an active participant in those ill-fated decisions and mistakes. So they deserve to bear more of the responsibility.
See here for pre-release addenda, all that's available on the public internet this moment:
https://www.reddit.com/r/pittsburgh/comments/dv9j08/what_are...
Now, if yinz'll excuse me, the driver in the opposite lane going straight has out-of-state plates. So, I need to stop typing this message to peel out / teach them a valuable lesson about right-of-way when the stop light changes in Yinzerville - as I head to a city-government-sanctioned hazard-light parking spot directly in front of Primanti's to pick up my slaw with fries and sandwich garnish (and sugar with coffee and cream).
Inb4 the bicycle crowd arrives to tell me that the average car driver commits 50 felonies a minute.
Sure. As long as Tesla itself, the company and its leaders and its shareholders, are held personally liable for any major autopilot or FSD fuckups.
Be it a monetary ticket or time in jail, have them all serve that responsibility each and every time. And if a long stint in "jail" - i.e. not being able the conduct business of any kind for that period - is too traumatic for the company to survive, then maybe they'd rethink their level of recklessness.
And certainly, if I'm in any sort of roadside screwup nearly regardless of fault, my insurance rates are going up a lot, for a long time. So too should that be the case for Tesla and their products whenever and wherever their software makes a decision that is involved in or the cause of the same.
Are you sure?
That would include everyone who owns S&P 500 index funds, probably most (if not all) pension funds, etc.
Fund managers? Triply so. They 1. knew better 2. enabled that 3. with other people's money
The entire issue of excessive CEO compensation for the sake of driving stock price, while being completely divorced from any repercussions of any knowingly harmful actions is exactly the capitalism part of "late stage capitalism" that sucks.
Let's restate your question -- Why should shareholders not get to benefit from blatant disregard for truth or safety without any liability on the line when that stance results in collateral damage?
edit: Upon looking up some stats I might have misremembered, in any case the median human drivers is definitely vastly safer than Autopilot.
Try to find the number of miles driven or crashes in the report, you know, the numerator and denominator. You will not find it because they seem to think that a “safety report” is a single number. You would fail a 6th grade science fair project if you did that.
And they seem to think it is okay to deliberately publish such a report for two ton machines with software that has killed dozens of people. They are either so stupid they should not be allowed near a safety-critical device or they are being deliberately deceptive to a level that would make Elizabeth Holmes blush.
So no, their safety report is anti-evidence for safety. Only somebody with something to hide would publish such a deficient and deceptive report.
lol, if you knew about it, then why is it on the road?
You'd need a simulation that captured all the edge cases of the real world, and that's logically impossible.
In safety critical systems design specifying operational domain is quite literally one of the first things that are done, even before any serious work on the design starts.
The only reasonable phase where safety critical systems can be interacted with closely monitored public are acceptance testing that either validates the model or drags the team back at drawing board.
Yes they do. When you get your license you have to past a test that talks extensively about defensive driving and how to deal with other drivers.
Also, why wouldn't they be able to test it in a simulator?
And this is more like a staging environment than it is production—production is when you put it in the hands of customers, this is an internal release available only to employees.
I'm firmly opposed to FSD beta, but this video doesn't concern me at all.
Why not? What's the limiting factor? Video bandwidth?
We can run video games at 4k 60fps. Is that not enough?
Even if you could perfectly simulate conditions for the sensors to pick up (incl. precipitation, light, dust) you cannot possibly simulate all possible scenarios that can happen in the real world.
Instead you run some basic must-pass scenarios, like the one in the video, where there is a red light in front of the car, and a left turn green, in clear conditions and daylight.
If the model doesn't pass, then don't ship it!
Yeah, that's what integration testing is for.
I would be surprised if they couldn't test the brains of FSD independently in a simulator. That's just QA 101.
You can if you invest in good simulators. See: https://waymo.com/blog/2021/06/SimulationCity.html
> One way we ensure statistical realism in our simulated world is by creating realistic conditions for our autonomous driving technology to experience. So, if the Waymo Driver is going through spring showers in Detroit at sunset in SimulationCity, we can recreate raindrops on our sensors and even simulate other minute details such as the dimming light and solar glare.
If they can simulate rain drops, it's definitely possible to simulate traffic lights. They are a basic scenario for a self driving car. What this video highlights is the extreme lack of validation culture within Tesla FSD.
We can't expect people to never test self-driving cars on public, populated roads unless we are just acknowledging that we don't believe self-driving cars should be a thing, because I sure as hell am not comfortable with the idea of cars going straight from simulation to production.
This is what you're advocating for by saying people should test these things on public roads. That is production, none of the other drivers have consented to be part of the "test".
Tesla FSD routinely fails at 1 in 100 mile safety-critical failure modes. The simulation does not need to be perfect, it just needs to be 100x better than what they have and FSD needs to be 100x better before they should be allowed to inflict it on general consumers.
If they want to test with trained safety drivers in a robust and well-controlled real world environment to identify failure modes that need to be addressed, that is fine. But non-criminal usage by consumers is at least a factor of 100x away.
Any version of SDC software that has a stoplight bug is proof that the QC process of that software is broken and that the makers don't care about safety.
Do you have examples of stoplights that aren't red circles? Or do you have examples of when a Tesla approaching a glowing red circle should not recognize it a stoplight?
I built a CV system in 2013 that detected vehicle makes/models in varying weather and time of day. It was a lot harder than stoplights, we had a team of one, and it was with technology that is ancient by today's standards. Tesla has no excuse.
Most Teslas have multiple cameras, so the viewing angle shouldn't matter as much to the vehicle as it does to a human.
1. https://twitter.com/RealDanODowd/status/1695268712910348471
I was just commenting about that.
Is he really driving if he isn't actually in control most of the time?
(Source: I think I did do this once or twice.)
One of the things they talk about in the video is the hoops they had to jump through to get the car to make a complete stop at stop signs, since almost no human drivers actually do that.
The left turn arrow turned green, the car inched forward a little, and Elon immediately put on the brakes. The situation was no more dangerous than when a human driver mistakes which light is theirs and does the same thing, which happens pretty regularly in my experience.
And that this is a test version of the software isn't irrelevant, it makes a huge difference—I am much less opposed to internal company testers who know what they're doing than I am to a public beta in the hands of people who believe Tesla's (really egregious) marketing.
Which category do you think Elon Musk is more likely to belong to?
When cars first debuted, they were incredibly dangerous and killed lots of people. Yet we persisted.
Airplanes, too.
Fully autonomous cars are on the streets of San Francisco. They run into trouble, yet the experiment persists.
We should allow for civil suits, especially when negligence is found, but the only way to progress forward is to make real world attempts.
The promise of autonomous driving is worth trillions of dollars for all of the opportunity it will unlock. And as macabre as it may be, there will continue to be deaths along that development path.
For example, the hand crank was known to be fatal and pushed the founder of Cadillac, Henry Leland, to get it rid of it when it his friend was killed after helping a lady on the side of road.
https://www.studebakermuseum.org/blog/the-1912-cadillac-a-se...
Similarly in England and Wales deaths from horse-drawn vehicles were around 70 per 1 million people per year in the early 1900s. That's in the same ballpark as motor vehicle deaths in the 1980s and '90s (80-100 per 1 million people per year) [2].
(It should be noted that medical technology is better now. A lot of those 1900 fatalities would probably have been preventable if they had current medical technology).
A lot of people on HN seem to view the horse era as some sort of idyllic safe, quiet, and unobtrusive pedestrian and rider paradise. It was not. From that second article:
> It is easy to imagine that a hundred years ago, when cars were first appearing on our roads, they replaced previously peaceful, gentle and safe forms of travel. In fact, motor vehicles were welcomed as the answer to a desperate state of affairs. In 1900 it was calculated that in England and Wales there were around 100,000 horse drawn public passenger vehicles, half a million trade vehicles and about half a million private carriages. Towns in England had to cope with over 100 million tons of horse droppings a year (much of it was dumped at night in the slums) and countless gallons of urine. Men wore spats and women favoured outdoor ankle-length coats not out of a sense of fashion but because of the splash of liquified manure; and it was so noisy that straw had to be put down outside hospitals to muffle the clatter of horses’ hooves. Worst of all, with horses and carriages locked in immovable traffic jams, transport was grinding to a halt in London and other cities.
and
> Motor vehicles were welcomed because they were faster, safer, unlikely to swerve or bolt, better able brake in an emergency, and took up less room: a single large lorry could pull a load that would take several teams of horses and wagons – and do so without producing any dung. By World War One industry had become dependent on lorries, traffic cruised freely down Oxford Street and Piccadilly, specialists parked their expensive cars ouside their houses in Harley and Wimpole Street, and the lives of general practitioners were transformed. By using even the cheapest of cars doctors no longer had to wake the stable lad and harness the horse to attend a night call. Instead it was ‘one pull of the handle and they were off’. Further, general practitioners could visit nearly twice as many patients in a day than they could in the days of the horse and trap.
[1] https://legallysociable.com/2012/09/07/figures-more-deaths-p...
[2] https://www.bmj.com/rapid-response/2011/10/31/cars-and-horse...
Back when we used horses for everything, people used to walk aimlessly in streets and children used to play in them. We advocate against that now.
Society chose a new mandate. It wasn't just the auto companies - it was all of us.
“We should kill people because it will unlock market opportunity”, is why we need to start charging the people in charge of these companies with murder when they inevitably do just that. I don’t care about Tesla stock, I care about not being killed by one of Musk’s “hype” projects.
There is absolutely no reason you should assume "testers" know what they are doing. I have met plenty of people with decades of experience in "testing" barely know what they are doing. Even in the case they know what they are doing, they shouldn't be testing a *deadly* vehicle with potentially broken software on heavily populated *public* roads.
This is another way of saying that self-driving vehicles shouldn't exist at all. At some point we have to test them on public roads, preferably before putting the software into the hands of regular users. If you ban even internal company testing, then what you're saying is that self-driving vehicles should never exist.
> This is another way of saying that self-driving vehicles shouldn't exist at all.
It's another way of saying that self-driving vehicles should be invented somewhere else, so that in 10 years you have to beg and plead for an overpriced second-rate implementation with a worse safety profile.
It's not surprising that it made a mistake during testing. That's what testing is for: rooting out mistakes.
Testing is for rooting out errors. Validation is for proving you achieve the desired specifications and failure rate. Validation occurs when you believe you are done or when testing becomes inadequate to discover failures. It is about “proving” the negative with respect to errors.
Broken, incomplete software where defects can be routinely discovered after light usage has no place being used by consumers on public roads.
No more weights: https://www.reddit.com/r/teslamotors/comments/zv5xp9/a_warni...
How strikes work: https://www.youtube.com/watch?v=otknSUQ4ICY
The inability to robustly solve simple, obvious cases and not failing safe is indicative of a sloppy development and validation process that is incompatible with the deployment of a safe safety-critical device.
It felt like more than a few inches. (I'm not splitting hairs here, I really do feel it was qualitatively significantly different from your description, which made it sound like the car was cautiously starting to move just a bit.)
> and Elon immediately put on the brakes. The situation was no more dangerous than when a human driver mistakes which light is theirs and does the same thing, which happens pretty regularly in my experience.
When humans make this mistake, they stop the car themselves. When this car made this mistake, someone else (Elon) had to stop it. These are not equivalent. In the former case you can argue no accident was going to happen, but in the latter you can't.
> that this is a test version of the software isn't irrelevant, it makes a huge difference—I am much less opposed to internal company testers who know what they're doing
This is on a public road. Try pulling this off (well, please don't) in medicine and see how it goes.
This is not nuanced enough. I've been in a cab to JFK in the snow and the driver was speeding around a turn that the car started sliding and eventually crashed into the side of the road.
There are now over 40,000 car accident fatalities in the US per year. So no, humans make mistakes that they cannot correct. https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in...
Note that these are based on auditable published statistics and are likely a overestimate of the risk as we must assume the worst when doing safety-critical analysis. Tesla could improve these numbers by not deliberately suppressing incident reports and by not deliberately choosing not investigate to avoid confirming fatality reports. But, until they do so we need to err on the side of caution and the consumer instead of the for-profit corporation.
[1] https://static.nhtsa.gov/odi/ffdd/sgo-2021-01/SGO-2021-01_In...
It's hard to tell because the camera is at such a weird angle, but from what I can see the vehicle remains pretty firmly behind the stop line throughout the entire encounter. We can quibble about how fast it was accelerating, but I regularly see worse false starts at lights.
> When this car made this mistake, someone else (Elon) had to stop it. These are not equivalent. In the former case you can argue no accident was going to happen, but in the latter you can't.
I'm actually more worried about the human case than the human+autonomous case. In the human case it is up to the entity that made the mistake to correct their own mistake. In the autonomous vehicle case you effectively have a second set of eyes as long as the driver is paying attention (which they should be and Musk was). This is why I say that it makes a difference that this was internal testing—the driver wasn't a dumb consumer trusting the vehicle, it was the CEO of the company who knew he was using in-progress software.
> This is on a public road. Try pulling this off (well, please don't) in medicine and see how it goes.
Requiring that autonomous vehicles never be tested on a public road in real world conditions is another way of saying that you do not believe autonomous vehicles should ever exist. At some point they have to be tested in the real world, and they will make mistakes when they leave the controlled conditions of a closed course.
Sure, but that's a wildly different case than "it's ready for public roads we super promise"
External intervention was required to stop the car from entering the center of the intersection.
That's not the same thing as "inched forward" is my point.
> In the human case it is up to the entity that made the mistake to correct their own mistake.
You're completely ignoring how likely these events are or how severe the errors are in the first place. You can't jus count the number of correction points and measure safety solely based on that.
> Requiring that autonomous vehicles never be tested on a public road
I never said that. (How did you get from "look at how it's done in medicine" to "this should never be done"?) What I do expect is responsible testing, which implies you don't test in production until at least you yourself are damn sure that you've done everything you possibly can otherwise. Given everything in the video I see no reason to believe that was the case here.
For me this is the important part. Too many people already drive far too distracted. Imagine giving many of those same distracted drivers a great reason to ignore what they're doing even more. Very frequently when I'm stopped at a traffic light I'll see the person next to me on their phone waiting for the light. Would we trust that someone doing that would lookup when their car starts driving into the intersection?
I really do want self driving cars to be a thing but I wish we were going about it a bit differently and I wish other people who wanted it didn't play things down as much as they do.
Isn't this precisely why self-driving (even if it isn't perfect) could actually make the roads safer. People, as you say, are already super distracted (partly because driving is boring).
When I say that the technology makes people feel like they can be less attentive I really do mean it. There was the SF tech worker who was playing candy crush or something when his Tesla smashed into a barrier on the highway. I have friends who own Teslas and talk frequently about how they like taking them on road trips because they can relax a bit more and pay less attention to the flow of traffic (it'll brake for you!!! they say). In a world where these cars have to share the road with human drivers and drive on roads that are under construction or in poor weather conditions I just don't see how we can say this is safe.
How attention-monitoring strikes work: https://www.youtube.com/watch?v=otknSUQ4ICY
Top comment: "The cabin camera really does feel super solid at detecting when I’m distracted. Even if I’m just like searching a song on the infotainment it will get onto me which is annoying but I completely understand and am glad that it works so well ..."
First, let's assume that perfect law-abiding, self-driving cars exist. One one hand they would eliminate incidents caused by inattentive driving, on the other hand they would in fact create incidents where attentive driver in the right of way would yield for an inattentive driver. Change in total incidents would depend on proportion of these events. Anecdotal evidence is anecdotal, but in my own experience number of incidents I have avoided simply yielding when having right of way is much higher then the number of incidents I have gotten into due to my own mistake.
Second, actual "self driving" cars are far from that, especially in their interaction with other drivers.
Third, there second order effects. E.g. a car quickly, unexpectedly maneuvering could cause other car to break sharp, which could end up in an accident where the original car is not even involved. With an increase of cars behaving differently than the local custom such accidents are bound to happen.
Most probably we are going to see an increase in the number of accidents with proliferation of semi-autonomous vehicles before that number starts to dwindle.
the attentiveness monitoring and strike system? If they ignore it, they will quickly exhaust their strikes and get locked out for bad behavior until they learn to take it seriously.
Just imagine: Constant 100% always-on supervision, supervision of the supervisors with 3-strikes you're out attentiveness monitoring, automatic and manual reporting of possible anomalies with full system+surroundings snapshots to inform diagnostics and development, immediate feedback of these into the simulations that validate new versions, and staged rollout that starts at smaller N (driving simulators are actually pretty good) and continues intensive monitoring up to larger N. Even Phase 3 trials only involve thousands of people, while FSD beta is driving a million miles per day with monitoring that feels more like Phase 1 or mayyybe Phase 2.
One day drug development will be this sophisticated, and it will be glorious.
Beta software shouldn't be on the road where other people can be impacted by it. Want to run into the back of a fire truck? Fine by me. Just do it on a track (and record it). I'll laugh from my computer chair rather than being potentially exposed to it on the road.
Early Access also doesn't attempt to run a red light because 'cars are supposed to go straight' when they're taking a protected left hand turn.
From one of the posts above. The monitoring system could not detect if a person was in the seat or not, and would autonomously drive with a giant teddy bear, a giant unicorn and a completely empty seat and repeatedly hit a moving object the size of a child.
>Except for the part where they can not even detect trivial cases. One of the easiest possible cases, a giant teddy bear in the seat “holding” the wheel (by attaching a simple weight to the wheel), is determined to be a attentive driver [1]. A T-ball and they strike out. The inability to robustly solve simple, obvious cases and not failing safe is indicative of a sloppy development and validation process that is incompatible with the deployment of a safe safety-critical device. [1] https://www.youtube.com/watch?v=CPMoLmQgxTw
Still, clever bypasses need to be taken seriously. Fortunately, Tesla has a track record of doing just that and closing loopholes: https://www.reddit.com/r/teslamotors/comments/zv5xp9/a_warni
A bypass that requires completely vacating the driver seat is probably not easy enough to be widely exploited. If you have evidence to the contrary, please do tell, but while we are exchanging memes, let's review the Green Hills Software standard of excellence, just to contextualize the shade they throw: https://www.youtube.com/shorts/UdSVMgay0MI
Elon had his hands on the steering wheel ready to make any intervention, this is hardly some crazy and dangerous testing going on. I would be far more worried about the average American driver than Elon testing the latest version of FSD.
I’ve had an incident where a crazy guy fully stopped his car on the I5 fast-lane to LA trying to brake test me before getting out of his car to threaten me. I’m far more scared of people than I am of software.
(have lost friends to both drunk and distracted drivers)
2. Elon mode. https://www.theverge.com/2023/6/20/23767041/tesla-hacker-elo...
3. https://twitter.com/RealDanODowd/status/1687106791715573761
4. The video clearly shows no hands are on the wheel at various points when Elon lowers the camera in his hands or when making 90 degree turns so the side of the yoke is pointed up and visible with no hand on it. The video is literally direct video evidence that your claim is untrue and you have no idea what you are talking about.
In the "RealDanODowd" videos he has fitted an aftermarket accessory to the steering wheel to fake having someone touching and the other video "Elon Mode" had someone literally hack the car. Having Elon himself make a publicity video isn't a reflection on how all the cars work.
Anyway, Ford lets you use their system without hands on the wheel, as do the majority of other manufacturers. People just want to hate on Elon because of the "anti woke" crap but there's nothing demonstrably different to any other auto maker.
Way back in 2019 I rode in a Honda Civic that had lane keeping functionality (auto steering and braking) which let you take your hands of the wheel for short periods of time. I imagine the tech was about 1000 times less sophisticated, and in fact, I remember it leaving the lane and needing correcting often, where were the scary and inflammatory articles then?
Let's get real, there's one rule for any news about Tesla, and another rule for other car makers.
I said that Elon does not have his hands on the wheel. You then retorted saying that it will not let you drive without both hands on the wheel, implying that what I said is impossible.
I then provided 3 independent techniques available to Elon Musk that would allow him to drive without his hands on the wheel and pointed at direct video evidence proving your baseless assertion and implication are false.
As to your second point, Ford’s system is explicitly designed, advertised, and declared to be safe to operate hands free with attention being determined through a camera driver monitoring system (DMS). Tesla explicitly declares in the legal fine print that their system should only be used by a attentive driver with their hands on the wheel.
If you think it is unfair to hold Tesla to their own standards for what is safe usage then take that up with Tesla. If they just take liability for hands-free operation like Ford then Tesla can remove that requirement and there would be nothing to complain about if Tesla shows hands-free operation because that would be a legally supported mode. Until they do so they are absolutely guilty of demonstrating unsafe and unsupported operation in official, sanctioned, advertising.
I don't ride a Telsa or an Elon and I know that's not true. SMH.
Are you trying to imply that a human driver monitoring a self driving system is somehow increasing the risk to others when its actually decreasing risk?
The question is what % of drivers will fall under 1 vs 3?
It’s called Full Self Driving. That matters, even if they clarify in the fine print.
They are selling a software packaged called “Full Self Driving” that includes some current features, some “beta” features, and promises of additional functionality.
Yes, Tesla has not yet released any non-beta feature that is really “Full Self Driving” but the marketing can be pretty muddled on what the car does today vs what it may do in the future.
The FSD computer required for the latter ("HW3") is a real physical part: https://i.imgur.com/WSWzOGQ.png
The HW3 situation was Tesla updating the hardware for people who both bought an older car and paid for the FSD "preorder" package, since they later determined that the computer shipped in those older cars was not enough for upcoming functionality, despite saying in 2016 that all of their cars had the hardware necessary for Level 5 Self-Driving.
https://www.tesla.com/blog/all-tesla-cars-being-produced-now...
> Tesla CEO Elon Musk says the new hardware will support Level 5 autonomy, which means the cars can drive themselves completely. However, Tesla will not enable full self-driving immediately. Instead, Musk says the company will roll out features to enhance safety and assist drivers, eventually leading to full autonomy.
https://www.cnet.com/roadshow/news/elon-musk-says-all-tesla-...
So they've been selling a product called "Full Self Driving" since 2016, and also claiming along the way that the current hardware in the car is capable of "Full Self Driving" in the future, pending safety testing and regulatory approval.
They also have a marketing video that says "The person in the driver's seat is only there for legal reasons. The car is driving itself" that has also been up since 2016: https://www.tesla.com/autopilot
Also people who had initially opted for just enhanced autopilot on HW2.5 then later decided to upgrade to FSD.
> So they've been selling a product called "Full Self Driving" since 2016
They've been selling hardware capability (hence the HW3 upgrade for those who bought it) for when FSD eventually releases. To me it seems reasonably labelled that it's future functionality and not yet FSD.
Either way, FSD had not released at the time and as far as I can tell the allegations target regular autopilot, not about being mislead into preordering FSD capability.
A big part of a valuation of the company is that Tesla can continue to flout any rules and do whatever they want, while blaming anyone and anything in the way. Their valuation assumes that services like self driving will drive P&L.
If the company is held accountable, you have to factor in the cost of that. Not good for a stock as highly valued.
Certainly! American juries have been known to return multi-billion-dollar punitive damages when they're convinced a company has acted maliciously. Look at JNJ, which is on the hook for $9B for selling baby powder contaminated with asbestos.
Note that even in cases where the company being punished was the company that governments cared about, regulators usually aren’t too shy about this. US and European governments are big Microsoft customers, say, but both US and EU regulators have punished MS for bad behaviour in the past (and, particularly in the EU case, forced it to change aspects of its business practices).
Says the company that makes a product literally called "Full Self-Driving." What a bunch of fucking lowlife charlatans. I surprised they didn't sign it with a smiling poop emoji.
I only use Autopilot on motorways in the UK because I don’t trust it on single carriageways. Even on motorways roadworks can scar the carriageway in such a way that temporary lines are still visible after the works are visible. With the right lighting I definitely see things that would confuse a vision only system. Scary stuff.
> The 2019 crash, which has not been previously reported, killed Lee and seriously injured his two passengers, including a then-8-year old boy who was disemboweled.
This hit home really hard. My son is 7.
Annoyingly, to prove you are in control you must apply slight pressure on the steering wheel. I say annoyingly since it sometimes doesn’t register and you have to give more, which is counterintuitive if it is doing the right thing.
On single carriageways this is worse since you are effectively trying to steer the car into oncoming traffic or off the road to prove that you are watching. That’s one more reason I don’t use it on these types of roads.
All-in, it’s not really worth it tbh, and this is just the basic autopilot.
Maybe that's how you're "supposed" to use FSD but realistically nobody does that. Teslas are not super common in my area but I have one friend who has one with Autopilot and uses it on his ~35 minute highway commute to work. He reads his email on his laptop next to him and doesn't look up at the road until his car gets off the highway. That's how 99% of people will use this.
Your friend is pretty reckless.
I've personally never felt like I couldn't take control in <1s.
Yes, more. This is a fundamental problem with these sorts of “computer does safety critical thing but can’t do it perfectly” things; you need someone whose job is to do _nothing_ most of the time, but have full awareness of what the machine is doing and be ready to stop it at a moment’s notice. Humans are, generally, quite _bad_ at doing mostly nothing but remaining super-alert.
My hypothesis for the Kia is that direct sun on corners at speed might affect the vision and/or collision system[0].
Is an infrequent jolt a reasonable trade off for the benefits? As an experienced driver o think so, but would my kids be able to handle it when the learn to drive?
0 - https://www.kia.com/content/dam/kia2/in/en/content/ev6-manua...
Have you used the system in these situations?
There's been a ton of construction on the freeway (motorway) near me that's left several visible lane lines, and the car has always been dead set on the actual lane somehow. It still drives (acceleration, steering) like a drunk 10 year old going to the gas station for a Slurpee, but the actual lane recognition aspect of it has been impressive to me.
Pretty good description from my experience as an owner for 4 years. It also had so much variability in behavior and regressions. Sometimes it would be incredible and maintain lane even in confusing poorly marked lane lines. Other times it would panic alert at every HOV on/off ramp for a 30 mile drive. Sometimes both of these things in the same drive!
Amazing that they charge for it then?
> Self-driving capability is central to Tesla’s financial future, according to Musk, whose own reputation as an engineering leader is being challenged with allegations by plaintiffs in one of two lawsuits that he personally leads the group behind technology that failed. Wins by Tesla could raise confidence and sales for the software, which costs up to $15,000 per vehicle.
The $15k software is Full Self Driving, which is still in beta and has very clear customer warnings and consent screens before using.
But the initial versions of this beta software wasn't even released for consumers until October 2020, a year after the crashes.
It’s almost as bad as gun people arguing about gun minutiae. Was it autopilot? enhanced autopilot? Vaporware FSD that was sold but people didn’t realize does not exist? Real FSD that is in “beta”?
Just like how reporting of some dude shooting up a nursery school will get derailed about mansplaining semi-automatic vs burst fire, anything Tesla must dissemble through the various bullshit.
You're basically correct.
The price makes this so much worse. If it were free and plastered in warnings, people probably wouldn't trust it. But because people first pay $15k for it, they now have 15 thousand reasons to want it to work and want to trust it. Paying for it creates a strong cognitive bias for disregarding any warning.
And 15K may not be a fortune for some people, but wealthy people don't get that way by being frivolous with their money and a 15-35% bump in the cost of a car is definitely going to be noticed.
The claimants here could well have bought a FSD package even though they only got access to EA features. Either way, a loss here that shows Tesla knowingly lied about safety features or negligently didn't fix known problems would probably hurt FSD sales, even of the specific judgement only relates to EA.
It is crystal clear that Autopilot, Full Self Driving, and other marketing terms are deceptive and dangerous, yet led countless people to spend five figures each enriching the now wealthiest person.
There has to have been some financial incentive for key regulatory personnel to look the other way so frequently. I don't mean Elon dropping off bags of cash, I mean something as simple as them having Tesla shares, but not being forced to disclose this or recuse themselves from any decision or case relating to Tesla.
> the Autopilot system caused owner Micah Lee’s Model 3 to suddenly veer off a highway east of Los Angeles at 65 miles per hour, strike a palm tree and burst into flames, all in the span of seconds. The 2019 crash, which has not been previously reported, killed Lee and seriously injured his two passengers, including a then-8-year old boy who was disemboweled.
> Wins by Tesla could raise confidence and sales for the software, which costs up to $15,000 per vehicle.
In this case it's the consumer that's made to look ridiculous:-
You can buy the car and get wrapped around a tree. But if the courts don't think Musk should pay you damages for that, then somehow you're more likely to buy the car.
> the company has argued that Lee consumed alcohol before getting behind the wheel and that it is not clear whether Autopilot was on at the time of crash.
Given that this case hasn’t been previously reported, I don’t think anyone can claim to know that Tesla’s version of the story is false. Presumably if they’re confident in it, Tesla won’t agree to a last minute settlement, and we’ll see what evidence comes out in the trial.
If anything I would expect the opposite; if the manufacturer were found liable in a few of these cases but continued to sell the thing, that would make it a more attractive product (Mercedes has a limited self-driving thing where they take some responsibility up-front, say).
Currently, when you use "self-driving", you need to pay more attention not less, because the system adds an additional unpredictability calculation to your driving, that you wouldn't have if you were just driving in the first place.
Don't get me wrong, I'd love to have a car I could tell to go the shop and get serviced. Or come to the airport to pick me up. Do I think that could happen? Maybe sooner with infrastructure support (but that'll be expensive), so realistically I'm pessimistic.
Why is this the future? Can we make it not the future? If the benefit of switching to electric is that high and the bar seemingly low (given the above), then shouldn’t we be seeing way more fully electric vehicle companies and models?
Nearly every other luxury automaker has an automatic steering and lane change feature as well.
I think that other automakers will be hit with similar lawsuits regardless of what the driver assist system is called, because it will be easy to argue that the driver assistance systems assume some percentage of responsibility while active, and because you don’t have to win a lawsuit to reach a favorable settlement.
Tesla may pay out more than other automakers due to its optimistic marketing.
Because these threads always become Tesla hate instead of an objective analysis.
All of these lane centering ADAS (including Autopilot) will kill you if you leave them unattended long enough (usually by taking a tight off-ramp at an unsafe speed). These is very evident to people who actually use these systems.
And there's nothing wrong with such behavior, it's an L2 system. Cruise Control will also kill you. These systems have been on the road for a long while and people aren't dying left and right because of it, they are empirically reasonably safe. They're probably a net positive as they likely drastically reduce rear ends.
People just get riled up at one off accidents because they make the news.
Are they calling smart cruise control "Full self driving" and "Autopilot"?
Smart cruise control has existed for a while now, and it is not regularly veering off course and slamming into palm trees & fire trucks.
Full self driving Teslas - according to Tesla - do not actually exist - and yet they sell & market them and have the outcomes that they do.
And then they have the balls to lie and say the system wasn't engaged - and tell everyone to keep using Full Self Driving Autopilot as normal.
How often does autopilot in airplanes speed up, veer off course, and crash into fire trucks and palm trees?
That’s… one term for it.
I also wish they’d knock it off with purposefully misleading marketing. They have a decent product but for whatever reason their CEO feels like compulsively lying about it.
It will just not happen as long as self-driving cars and humans share the same roads. AI will never be better at avoiding a playing child who accidentally stepped on the road as an average human driver.
If we want self-driving cars to become a reality, we should instead make separate roads, and let the drive there. Some analogue to railroads, that are used exclusively by trains.
What makes you so confident?
I'd argue for the reverse-- there is absolutely no way that humans are able to compete long term, because human reaction time and focus/attention is inherently limited-- as soon as computer vision starts to outperform, humans will NEVER become competitively safe drivers again (could be 5 years, could be 50, who knows, but this seems EXCLUSIVELY a matter of time, especially considering the progress we made in the last decades).
Very hard to predict what will be possible with AI in 10 years time, but I wouldn’t bet on humans here.
Humans have a number of disadvantages compared to an AI agent in this scenario - they look at phones, get tired, technically have slower response times…
And then AI has seen tremendous advancement in the last 3 years with it basically smashing a load of tasks that were thought to be entirely ‘non-automatable’ by many people.
Plus think about what has been achieved in the self driving space in the last 10 years and then imagine what can be done in the next 20. It’s probably not a 1-2 year thing (imo), it’s probably a 10-20 year thing (but again totally unpredictable! Maybe 5 maybe 50!).
And self-driving without operating cooperating alongside human drivers just won’t happen from an infrastructure perspective.
"On Monday, Waymo began letting the public pay for rides in its driverless cars in San Francisco."
https://www.nytimes.com/2023/08/21/technology/waymo-driverle...
Had the problem been tackled more intelligently we could have had closed off "self driving lanes" or entire thoroughfares in major cities with advanced sensors built-in and no interaction with humans at all. Considering how many hundreds of billions of dollars have been invested into the problem, we would definitely have had more to show for it by now.
Because safety is a cost, not a feature.
Air travel and sea travel did not get safer because some companies got together and decided to spend a little money to advance society. Ships sank and planes crashed frequently.
Things got safer because governments took action. Look at the 737 MAX. The cause of the crashes wouldn't have been investigated and Boeing wouldn't have fixed their plane if governments hadn't taken action.
At scale, companies do not value human life, so regulations add costs to change the incentives. Otherwise human life is just another tragedy of the commons.
Do we know anything about the likelihood of criminal charges over that crash?
"Why urbanists are wrong about autonomous vehicles w/ Tesla critic Ed Niedermeyer" 2023-08-17
https://micromobility.io/news/why-urbanists-are-wrong-about-...
TLDR: Equating autonomous vehicles (AVs) with Tesla's FSD is a category error. Tesla's FSD is a fraud.
If you extrapolate out the progress to around 5 years I see a world that has car insurance companies giving you a huge discount to use self driving. Maybe insurance companies will have a lower deductible while having a crash during self driving vs a high deductible during manual driving.
I see FSD getting so good in 10 years the government would mandate it because it would eliminate a good portion of the 40k traffic deaths a year.
It’s pointing out correctly that the Mercedes system is the only one that allows you to take your hands off the wheel.
Death is inevitable in literally any system, the point is to lower the death probability to well below human drivers.
Probably because the first sentence tries to say that 'full self driving' does not mean 'full self driving.'
That describes just about every civil court case in the world. The law is all about interpreting the meaning of words. Whether Tesla misled consumers into thinking their car could do more than it was capable of is a very important ruling, and will set a strong precedent for the entire self driving industry.