Tesla’s self-driving technology fails to detect children in the road, tests find
theguardian.com
theguardian.com
Once FSD seems ready for level 4 or 5 automation, then I'm sure governments will have Tesla prove all sorts of things about how it functions and how good/safe it is before it is approved.
People learning to drive are not equivalent to a closed source driving AI.
If their customers are actually driving instructors for their cars then Tesla should be employing them, verifying they are competent at the task, and paying them the prevailing rate of ~$25 per hour. They should not be advertising it as a product that is fit for purpose when it requires constant supervision by trained professionals.
We don't. Over here driving school is several months of training by licensed drivers in cars with dual control so the teacher has full control over the car in case of an emergency. the idea that your Uncle Bob teaches you how to drive would probably be considered insane. The traffic related death rate here is also like 4x lower than in the US. This kind of stuff isn't normal.
Having computers that make video games from the 90s look bug free drive two tons of steel around at lethal velocities is bonkers. Public roads should not be playgrounds for 'beta tests'.
I'm surprised they haven't coded in an easter egg where the screen displays an animation of Wile E. Coyote being squished into a pancake when it strikes an object.
Can't wait for Elon to tweet out "will be fixed in FSD Beta x.y!"
sigh...
They sent some fixes to Tesla and advised them.
A few hours later, "Thanks so much for this, everything is working all great now."
"??"
"We flashed it onto a car and took it for a short drive, and couldn't see any more issues".
Same with (one of) the phantom braking issues. Yeeted a fix over the fence in a day or so, which "fixed" some of them, but introduced different ones. Yeeted another fix a day or two later that fixed some of the different ones but re-introduced some of the original ones.
Anyone who can tell you with a straight face that that is a safety-first culture is lying.
But its hard to say, since noone seems to have tested that publicly. Here's an article that gives a fairly balanced look at some of its other failure modes, though: https://www.caranddriver.com/features/columns/a36118292/supe...
"It made it 8.8 miles before abruptly yanking the wheel left, toward the median, and canceling."
Nothing about this would keep me from using it. I'm sure its a great system, but like all of them, it requires attention. If you don't intervene (as it expects you to) accidents can happen.
I thought Tesla announced something like that a while ago, was it just loose talk on Twitter by Elon?
An automated system to prevent harm caused through parental negligence would be interesting, but how would you even detect that a baby was present? Maybe some kind of 'have' token attached to the carrier or car seat? OTOH that could cause more harm via accidental strandings. Would be interesting to search for an elegant solution.
It failed to detect a stationary mannequin in one specific scenario. Does this generalize? Does this mean there's a major safety problem? These would be assumptions that this test can not prove.
A comprehensive safety analysis has to also include these scenarios: https://youtu.be/hx7BXih7zx8?t=205
I can't fugure out the "even slow down" claim when the car goes from 40 to 25mph by itself?
But it did not slow down anywhere near the mannequin, and even sped up a little bit when hitting the mannequin in 1 trial.
The Dawn Project test cannot answer such questions.
I believe that NHTSA is working on this and related questions across the industry and I expect them to produce a more useful result on their own than what can be produced by single-minded reports like this.
I'm struggling to see how any generalization would not be a much worse scenario.
No. But there could be protesters blocking the highway. As annoying as they are, running them over is not an option.
They said FSD failed but it wasn't even on.
He also used to go on about the "linux threat" of people using linux rather than his software https://www.eetimes.com/odowd-then-and-now-on-linux/
https://twitter.com/TaylorOgan/status/1478802681141645322 (This is January 5th 2022)
Same author, same test, months later:
https://twitter.com/TaylorOgan/status/1556991029814886404 (This is August 9th 2022)
The difference? There is absolutely no improvement.
Expecting that the FSD (Fools Self Driving) beta software would get better over time as promised by Elon, this video shows that it doesn't look good at all with this latest failure which drivers on the road, and even pedestrians were no better off and are no safer with these updates.
It is not early days anymore and even with these updates, it is still an unsafe contraption which not only has been falsely and deceptively advertised by Tesla, it is not even near the claims of the robo-taxi level of FSD (Level 4 - 5) with the number of price increases over the years with a product which doesn't work looks more like a scam.
At this point, FSD is essentially Fools Self Driving.
Which competing EVs do you like and why?
Nobody is going to tolerate FSD vehicles bugging out and randomly driving into parked cars or pedestrians, killing people in the process.
You can tout millions of miles all day ever day, but it doesn't matter. When these vehicles are put into even semi-tricky situations, they fail far too often. Tesla's software doesn't even require tricky situations... it'll just randomly drive you into a wall or fire truck without warning.
These systems however, are not.
I think most folks would rather not be killed because Elon's ego decided a decade ago that Tesla would never consider Lidar sensors, for example, despite the mounting evidence simple cameras are not sufficient.
Recall the Toyota stuck accelerator pedal issue years back? There are people who still refuse to own a Toyota.
What about the Boeing 737-MAX fiasco? An entirely software related bug caused a statistically trivial and insignificant amount of people to lose their lives - yet today there are still people who will never fly on a Boeing aircraft again, despite the issue being resolved.
People will not tolerate loss of life from these systems. They must be substantially better than the average human for the public to be interested.
This determination needs to be made with data. As far as I can tell there is no conclusive evidence that Teslas on autopilot are involved in more crashes per mile than human drivers, and some evidence that it is safer, though that comes from Tesla, so not giving that much weight. So either it's somewhere close to as safe as human drivers, or data collection in this area is just really bad. I don't know which.
The burden of proof is on the FSD companies, not average drivers. The likes of Tesla have to prove their software is far superior to human drivers - and so far they have failed to do so.
There are far too many FSD beta videos out recorded by real users that demonstrate obvious safety issues. The software is just not there yet, and may not be there for a long time, if ever.
Articles like this come out far too often to just be discarded as media "hit" pieces.
Boeing fixed their management/culture problems?
The states of the average driver also include drunk and drugged drivers. Texting drivers. It need to be way better than the average driver until a driver that does not do those things gets a safety benefit.
I would expect Tesla FSD to behave worse than humans when there is a vehicle malfunction too.
Are all driving miles equivalent? Do you think accidents that occur with FSD enabled are the same as where they most often occur under normal human control? What proportion of miles are driven in sunny US states?
Which metric is proposed?
Is this an argument supported by data? What about a sound experimental design?
Perhaps we could look to medicine (instead of automotive engineering) for guidance regarding which reductions in stratified accident incidence rate are significant and warrant continued investment in sensor fusion and assistive AI?
Tesla on the other hand is trying to make a mass-market version work right off the bat, even if it means debugging on the streets. Tesla will get a lot of bad press, will kill a few people, and the public will become sceptic about the tech. I think Elon Musk's approach is not just wrong, it is going to make it harder for Waymo and Cruise by making consumers distrust self-driving cars in general.
* This video was made by the The Dawn Project, whose goal is literally to make Tesla's self driving illegal. Their current goal isn't to make all unsafe self driving illegal, only Tesla's.
* The Dawn Project is founded and operated by Dan O'Dowd, who is campaigning for senate. His biggest platform issue is banning Tesla's self driving. He's definitely getting the press coverage he wants as a result of this publicly advertised test.
* Dan O'Dowd is the founder and owner of Green Hills Software, which makes self driving software for car manufacturers, and has more than a dozen partners with deep connections to the automotive industry. Dan does not disclose this conflict of interest.
* Green Hills Software has Ford and Toyota as direct customers. Dan does not disclose this conflict of interest.
* The Dawn Project explicitly outlined this test as "a small child walking across the road in a crosswalk" and it fails in both of these goals - the "child" isn't walking and the road isn't marked as a crosswalk.
* There is zero coverage of trials where Tesla did successfully brake. The test circumstances are clearly setup to make it fail. While noteworthy they were able to find the right conditions, not disclosing the work that went into making the test scenario only further fuels the bias of this test.
* It is unclear to me how they managed to get a Tesla to work in full self driving mode while plowing through clearly marked parking spaces in a parking lot. These are clearly not conditions that anyone would be using FSD. This is further marked by the fact that the Tesla braked hard from 40MPH to 25MPH as soon as FSD was enabled because it knew going that speed in a parking lot and driving over road markings it shouldn't was a stupid thing to be doing.
* FSD was enabled only seconds before being introduced to the stationary mannequin.
TLDR: There is a massive conflict of interest here that isn't disclosed anywhere. There is a massive incentive to use this as advertising for his senate campaign. The test doesn't actually put in any baseline effort to replicate what they claim to be testing. They put the Tesla in a situation that no one would ever actually be using FSD.
> In several tests, a professional test driver found that the software – released in June – failed to detect the child-sized figure at an average speed of 25mph.
The car was 40mph when put into self drive mode, so the car was slowing itself down even though the mannequin, which looks like a cone to my eyes, wasn't moving.
Are these people legit but counterproductive to their claimed goals or entirely bogus?
Ah, prominent mentions of LIDAR suggests it's just an advert disguised as a consumer safety campaign.
> We Demand Software that Never Fails and Can’t Be Hacked
Or possibly they're just weird?
Yeah, pretty much. I'm sure they could find a way to cause similar systems from other manufacturers to fail. None of the deployed systems are actually perfect in the "never fails" sense.