Tesla vehicle in ‘Full Self-Driving’ beta mode ‘severely damaged’ after crash
theverge.com
theverge.com
Something doesn't add up about this report. Teslas are not fly-by-wire. Their steering wheel and brakes are mechanical. You can't turn the wheel without the wheels turning. It therefore shouldn't be possible for the computer to overrule the human. Considering that, it is unclear what intervention the driver actually did and what "the car by itself took control" means. My initial hunch is that the driver isn't being entirely truthful in an attempt to shift blame from themselves to Tesla. But even if that is the case, it wouldn't clear concerns about the safety of FSD.
The designs for each of these three are different and will - while make the resistance heavier when un-powered, are not quite the same between each other.
Are you sure you couldn't overpower it or were you subconsciously holding back because you were worried about jerking the wheel too hard and how that would impact the vehicle? If you have ever driven a car without power steering, there is a surprising amount of force required to actually steer a vehicle mechanically, but it is something every driver used to do so it shouldn't be that difficult even in the worst situations when the power steering is refusing to cooperate.
That's good. It's happened to me several times, and it's one of the reasons I'm not a fan of Autopilot. When it happens, the car is usually following a curve in the road. I've also had it turn off because I barely touched the wheel, and that usually happens on a straightaway.
I'm not sure if I could overpower it or not if I'm honest. Last night when it happened, it took a lot of force before it relinquished control to me. While fighting it I was definitely thinking about how it'd react once it "let go", which could have easily influenced how aggressive I was in applying force.
Regarding power steering - I have a 2006 Subaru STI that's a track-only car. The power steering pump began leaking years ago, so I removed it and the air conditioning entirely. It takes a lot of effort to steer that car, but that makes it pretty fun on the track too. A day at the track often means my upper body is sore the next day.
That said, I'm not sure how that's relevant to Tesla's FSD or Autopilot.
I've not dug into the mechanics of the Model 3, but it seems reasonable to suggest that for Autopilot or FSD to work, the mechanisms that support power steering can also be automated through software. Meaning, it's unlikely that there's a traditional power steering pump, and the automation is done with stepper or servo motors. It's more likely that it's done with a single motor. I could be wrong, of course.
That said, irregardless of how it works, it's reasonable to assume that that system has to be able to match or exceed what a driver is capable of doing, force-wise. If that's the case, then it seems totally reasonable that the car could prevent me from steering it manually if it wanted to.
My googling has also suggested that Tesla does have a pretty standard EPS system although it doesn't use a pump and fluid like older systems.
But the force required to do that is just not high. It's not. Just try this on an empty highway. Engage AP in a context where you're doing a lane change or something and hold the wheel firmly. It won't be remotely able to overpower you. You'll feel it tug and then it will chime and give up.
> it wouldn't clear concerns about the safety of FSD.
If you follow FSD review channels like AI Addict, you'll know that there's no "safety" in the system. The human must maintain control at all times, as part of the agreement with Tesla for using the system, because it is clearly unsafe, if left on its own.
I also tried the 'FSD' on freeways where it can navigate on autopilot. It's really feels amazing and takes a lot of burden of driving at high speed off of me.
However, recently I was going onto the acceleration ramp, and turned on navigate on autopilot, Tesla told me to confirm a lane change to the right, I did and it somehow didn't see a car to the right of me trying to change lane to its left. That would be to same spot I would have gone to. So I had to manually intervene and take control of the wheel myself.
Manual override isn't difficult at all though, and there are multiple ways to do it. It can be done through turning the wheel manually, toggle the right stick upwards, or hit the brake. So I think if you still pay attention (maybe not as much attention as driving a non-Tesla car) when driving a Tesla, it's going to be okay.
I wonder if this will paradoxically lead to Tesla's becoming less safe overall as the software improves, because errors will start to become uncommon enough that people really stop paying attention.
From the sound of it it is currently unreliable enough that most people know that they really do have to pay attention.
It has been assert many times without much actual evidence.
> I wonder if this will paradoxically lead to Tesla's becoming less safe overall as the software improves
Again, something that people on HN are often totally sure is true but I have never seen any actual evidence that is true.
The believe in this is based on some specific cases in the airline industry that I am not sure necessarily translate to driving.
Probably because it is so obviously true surely? Do you really doubt it?
We've already seen Teslas crash due to autopilot failures in ways that definitely wouldn't have happened if people were actually paying attention (e.g. driving into the side of lorries) so there is at least some evidence anyway.
Yeah. I do doubt it. The question is about if the fall off of attention is larger then fall of of amount incidents. And then how many of those incidents are of a critically where you can not overcome the not paying attention part.
Its not at all clear to me if its a net negative in security, in fact I would suspect the opposite.
> We've already seen Teslas crash due to autopilot failures in ways that definitely wouldn't have happened if people were actually paying attention
Not really. We have seen Tesla crash mostly in situation where they crash all the time already. And that analysis is prove of existence, not prove of significance and net negative security. You need to actually show that this happens MORE often if Autopilot is engaged.
I have seen 100s of videos were Autopilot avoids a vehicle cutting Tesla off for example. In many of those cases even a human driver would not have react so quickly. Most of those cases would not result in death but some could. You need to actually take all of that into account.
Who's wants to be their cars supervisor?
For me, it's either fully drive and enjoy your driving ... or full FSD and relax ... if it's somewhere in between, per Tesla, it's pointless (and don't get me wrong, lane assist and cruise control are handy, but they've been around for eons).
Fact is tons of people use lots of driver assistance.
It would drive me nuts to have to supervise an FSD car through a city or any situation beyond the above simple motorway situation.
You'll never convince me otherwise. I don't want to be my cars manager.
Really, it's criminal at this point.
Imagine using it when driving at night. The most dangerous time to drive.
No thanks and no deal to safety critical 'beta' software.
Although saying that I think the issue is more that the action is slower than the reaction taking longer, if there's a difference...
These things will happen, no one can write perfect code after all. But I'm still very optimistic. Right now we're in a weird Twilight Zone, where you're expecting drivers to be able to take command from an automated system at a moment's notice. Odds are the driver was zoned out or texting his partner or something.
I have no idea how you could come to the conclusion that FSD "just needs to be better than a distracted speeding drunk driver".
You can drive outrageously bad, and even when you hit something you typically aren't banned for life from driving. An AI wouldn't keep inching forward at a red light and rear end someone. Even with what we have now with Waymo is much better than the typical driver.
Careful- this is true if interpreted juuust right, but it's subtle.
Ultimately, you care about, roughly, the net decrease in Bad Things (injuries, deaths, property damage, weighted however you want) (and possibly also the net increase in Good Things (people being able to get places, mostly; otherwise, an excellent FSD system is one which simply stays in park and refuses to move!)) that results from the deployment of an FSD system.
That looks roughly like the sum, across all trips, of "expected Bad Things if FSD was available as an option for that trip" minus "expected Bad Things if FSD was not available".
The key here is considering each trip as a choice between FSD and non-FSD and /then/ averaging. You can't just say "FSD's per-mile accident rate is lower, so it's safer"- that misses a lot of important effects!
For example, consider selection effects related to weather and road conditions. It's easy enough to imagine an FSD system that falls apart entirely in bad weather, but works alright (but worse than a human driver!) in good weather. Let's say it's got 0.1 Bad Thing Points Per Mile (BTPPM) in good weather and 100 BTPPM in bad weather. But human drivers also do worse in bad weather, naturally- maybe they've got 0.05 Bad Thing Points Per Mile in good weather and 10 Bad Thing Points Per Mile in bad weather.
Now, assuming the poor performance in bad weather is obvious to drivers, drivers will only activate FSD in good weather. So the measured BTPPM for FSD is roughly 0.1. Human drivers, though, still have to drive in bad weather sometimes - so you might see, say, 1 BTPPM from human drivers.
Even though human drivers are better than this hypothetical FSD system in all conditions, they look way worse!
And that's just one effect- here's a related, but distinct one. Take the same example from before, but now the FSD system matches the human in good weather and bad weather (0.05 and 10 BTPPM) and, in awful weather and road conditions, where the human might have 1000BTPPM, FSD manages just 100BTPPM.
The FSD system's performance matches or is superior to the human in all situations. Surely this leads to a net decrease in Bad Things?
Not necessarily- what if people take trips in awful weather at 100BTPPM which they wouldn't take with a human driver at 1000BTPPM? (And this assumes people aren't lulled into a false sense of security because they know FSD is "safe", leading them to underestimate the risk in awful conditions). This is where you have to weigh in "how much benefit does taking this trip in the first place provide".
And one more thing: remember you have to consider the vehicle. Any vehicle with FSD can and would almost certainly (if it's human-drivable at all) provide very sophisticated ADAS features when driven by a human, and would likely be new enough to have quite modern safety features. So you have to remember to not just think about "FSD accident rate per mile v.s. human"; you do have to make sure to compare within the same vehicle.
"Better than the average driver" could be interpreted to take all this into account, but the straightforwards interpretation ("lower accident rate on average") doesn't, and can be surprisingly misleading.