From the admittedly not much footage of Tesla FSD (1-2 hours total maybe) I’ve watched, it seems to be roughly on par with a student driver who occasionally panics for no apparent reason.
I do not believe that Tesla's technology will ever achieve "Full self driving" (levels 4 or 5, where a driver isn't required https://www.nhtsa.gov/technology-innovation/automated-vehicl...), and the labeling of their "conditional automation" system as "full self driving" is an absolute travesty.
People do honestly think they have bought a self driving car. The marketing lie absolutely tricks people.
Any of those could mean a car could randomly turn into the crosswalk.
Humans are horrible at assessing risk. We worry about Teslas because it's on the headlines now, but you're infinitely more likely to get ran over by one of the cited examples than a Tesla just by sheer statistics.
TBH I don't think it's ever crossed my mind whether a Tesla was in FSD or not, even when driving alongside one. As long as it doesn't enter my space, I'm fine.
Walking through Manhattan, I'm more worried about the human driver nowadays than the non-human.
Sure - which is why I said I agreed there should be some indication of it being in FSD or not, originally.
The problem is too hard for existing tools. Good job on the 80% case; let's look for problems for which the 80% case is good enough and back away from unsupportable claims about "just a little more time" bringing FSD in environments defined by anomaly and noise, in which the cost of error is measured in human life and property damage.
With self driving it takes longer for the one individual to learn, but the difference is that there is only the one individual, and it need not ever die. The learning is transferable between cars, regressions between generations need not be accepted. The problem of bad driving can finally be solved.
People say "how could it happen?!" and get angry for a software bug - that can be fixed - and seem to accept how drunk drivers do this (and a lot worse) literally every single day. But since it's a human, that's fine!
Edit: wording
I'm absolutely certain that they've attempted to simulate millions of hours of driving, yet this odd behavior happened in real life, which now can be studied and fixed.
If we never had the software on the wild, how would you ever really test it?
Genuine response:
1 don't let customers do the testing, especially if you don't train them (I mean real training about failures not PR videos and tweets and some small letter manual with disclaimers)
2 use employees, train drivers to test, have some cameras to check the driver to make sure he pays attention.
3 postpone testing until the hardware and software is good enough so you don't ignore static objects.
4 make sure you don't do monthly updates that invalidates all your previous tests.
Unfortunately it's always about the incentives and on a capitalist society the only incentive is money. So even if they could be safer, they wouldn't do it unless it's more profitable, specially being a publicly traded company.
After all, normal (complex) software tends to have lots of in depth details you need to test for; and a surface area that's pretty irregular in the sense that it's hard to do generalized testing. Some bits can be fuzz tested, but usually that's pretty hard. It's also quite hard for a generalized test to recognize failure, which is why generalized test systems need lots of clever stuff like property testing and approval testing, and even then you're likely having low coverage.
However, a self-driving car is amendable to testing in a sim. And the sim might be end-to-end, but it needn't be the only sim you use; the FSD system almost certainly has many separate components, and some of those might be easy to sim for too; e.g. if you have a perception layer you could sim just that; if you have a prediction system you might sim just that; etc.
And those sims needed be full-sim runs either; if you have actual data feeds, you might even be able to take existing runs, and the extend them with sims; just to test various scenarios while remaining fairly close to real world.
I'm sure there are tons of complexities involved; I don't mean to imply it's easy - but it's probably tractable enough that given the overall challenge, it's worth creating an absolutely excellent sim - and that's the kind of challenge we actually have tons of software experience for.
The onus is on the company trying to do this to figure out a safe way.
They don't (should not) get to test in production with real innocent lives on the line just because they can't come up with a better answer.
You could make a law that states that your FSD has to be at least as good as humans. That means you have the same post-problem verification but now the parallel with drunk drivers can be made.
The fact that enforcement is patchy is irrelevant — drunk driving is deemed serious enough to be an automatic infraction.
Also, most of the time drunk drivers are not actually that bad at moment-to-moment driving. That’s why almost everyone worldwide used to do it! You can still do the basics even when reasonably drunk. That doesn’t make you safe to drive. It’s still incredibly dangerous to do.
I think that some form of responsibility has to be assigned to the FSD in some way (the manufacturer? the user? some other entity or a combination?) regardless but I haven't found any clear case of how that would work.
It also makes me wonder how we would measure or verify a human driver with intermittent drunkenness. Imagine 5 seconds out of every minute you temporarily turn into a drunk driver. That's plenty of time to cause a major accident and kill people, but on the other hand that would mean that the combination of a situation where that would happen and the right timing to not be able to judge that situation has to apply. We do of course have the luxury of not having humans constantly swapping drunk and normal driving, so it isn't a realistic scenario, but it would make for a better computer analogy.
Besides drunk drivers we also have just generally crappy drivers that just happened to get lucky when doing their driving test (although there are places where no meaningful test is required so that's a problem in itself).
- drunk drivers are also not uniformly awful drivers: they can drive OK for the most part
- they still drive unacceptably poorly
- we strictly punish them on detection, before any potential accident
- drunk drivers and FSD have more in common with each other than competent drivers and FSD
- why is FSD not held to such a preventative standard?
One can argue that FSD is like a drunk driver driving a student training car with two sets of pedals and two steering wheels, and the Tesla owner/driver is like a driving instructor. But driving instructors are trained and paid to be quite vigilant at all times. Tesla play a sleight of hand and say it’s a labor saving technology, but also you need to be able to behave like a trained and paid driving instructor... that is a conspicuous contradiction.
And I’m ignoring future FSD capabilities because while I’d be happy for it to come about, we should discuss the present situation first, and I don’t believe it’s a good example where sacrificing lives now is acceptable in order to potentially save lives in the future.
The same can't apply to FSD because it isn't consistently 'driving drunk'. That analogy doesn't hold because it is not fixed software like a GPS-based navigation aid would be. Just like humans it does have more or less fixed parameters like the amount of arms and legs you have, that doesn't tend to change depending on your intoxication.
One could make the argument that it's not as much the "haha it is just like a drunk driver zig-zagging", but the uncertainty about the reliability. If a car with some autonomous driving aid drives across an intersections just fine 99 times out of a 100, and that one time it doesn't, that doesn't mean the car software was dunk 100% of the time.
Why FSD is not held to some standard, I don't know. I suppose that depends on how it is defined by local law and how the country it is in allows or disallows its use.
The problem with prevention and detection here is that like humans, the system is not in a static state. The trained neural network might be largely the same with every release, but the context in which it operates isn't, unless the world around it stops completely in which case two trips can be identical and because the input is identical the output can also be identical. Humans do the same, even if well-rested and completely attentive, knee-jerk reactions happen.
Holding FSD to a standard of a drunk driver isn't a valid comparison due to the non-static nature of the state it is in. This isn't even FSD-specific, even lane guidance/keeping assistance and adaptive cruise control isn't static, and those are based on pretty static algorithms. Even the PID-loops used on those will deliver different results on seemingly similar scenarios.
Perhaps we should stop comparing technology to humans since they are simply not the same. The static kind isn't and neither is a NN-based one. We can still explore results or outcomes because those are the ones that have real impact. And let's not fool ourselves, humans are far less reliable in pretty much every man-machine combination. But in human-to-human contexts we factor in those unreliabilities, and with machine-to-human or machine-to-machine we seemingly don't, which is pretty much the same problem you're describing.
This will be an interesting field of development, and if we simply take death toll into account, keep in mind that for some reason seatbelts were thought to have 'two sides of the story' as well when they were first introduced and later required. As with car seats for children and infants, a good idea might start out one way and over time (with the accompanying bodycount) it gets shaped into whatever we expect of it today. Same goes for aerospace, boats and trains, and that's even without taking software into account.
Certain medical conditions are analogous to this. This is handled in various ways, including doctors reporting you to the state DMV and suspending your license if your condition is not very well controlled.
On the other hand, it's not like the software is always bad and always in the same situation. That is a big difference with a human analogy; a drunk driver taking a trip in the car is drunk for the entire trip (unless it's a very long trip etc..), so would be impaired for the entire duration.
There are plenty of people and vehicles that are road hazards and are allowed to drive (or be driven) anyway, so if we really cared about that aspect on its own we could probably do with better tests and rules in general.
What I mean is that you don't see a headline on news outlets or twitter hashtags trending for every drunk driver fatality or crash that happens. Which for me tells that collectively, we are fine with it. We accept it as ordinary, as crazy as it may be.
I've certainly seen many more local "person dies in car accident" non-brand-related stories than "person dies in Tesla accident" stories.
Unpredictable death-robots roaming the streets are pretty novel.
I don't think that's very complicated, from a news perspective.
My comparison with drunken drivers is just with the regards to the odd behavior. If you look at a drunk driver (or even someone that fell asleep in the wheel) with a near crash, many times it would resemble this video. But the outrage from the public differs vastly.
That doesn't mean we need to endorse putting more cars on the road that sporadically act like drunks.