Tesla Autopilot thinks the moon is a yellow traffic light
twitter.com
twitter.com
Plus the cost of the latest FSD computer upgrade, even if Elon Musk promised your old model "has all the required hardware for Level 5 autonomy". Your fault for believing him, pay up.
* Isn't actually Full Self Driving
I've had a Model 3 for 3 years with FSD. My experience:
Autopark - trying to get it to show up is almost impossible. When it does show up, the chance of curb rash is between 95-99%.
Summon - for the most part works if it can connect. still does some sketchy stuff some times.
Smart Summon - literal piece of shit that will always do the wrong thing and then just stop in the middle of the road.
Navigate on Autopilot - kind of works, but it makes the auto-lane change feature finicky. Without NOA, auto lane change works fine, with it enabled, fails 80% of the time with the car jerking back into the original lane.
Can't wait for FSD though /s
So if you’re in Europe/Asia/Oceania or in the market for an S/X - you’re in luck!
I think stereo vision won’t help humans much at the distances we’re talking about (might be different for a car with cameras over a meter apart, but that will suffer from bad resolution compared to the human fovea).
Parallax probably is a clue we use. You want to brake gently to optimize comfort for the passengers, so let’s say you want to start braking 10 seconds away from the traffic light. Since sin(x) ≈ x, the image of the traffic light will move up about 10% every second at that moment.
On a camera, that could be less than 100 pixels, but I think that should be detectable.
⇒ you don’t have to discriminate between the moon and a tragic light, you just have to find the nearby yellowish blobs.
Is it publicly known what Tesla’s self-driving does? Given the above, I think I would try to extract a depth field from the known speed of the car and the camera images, discard or deprioritize whatever is more than 20 seconds out, and then let some ML algorithm loose on detecting points of interest such as traffic lights, cars, etc. (As opposed to “feed a ML algorithm lots of video, and hope it will learn to deprioritize stuff that’s far away)
A human disambiguates by looking around, lots of prior knowledge about how the world works, etc. That doesn't seem to be the case with the current Teslas.
When it's fully rolled out, it's going to kill a loooot of people.
Do you have any data to back this up?
Because there is definitely evidence that shows that it is saving lives: https://twitter.com/Model3Owners/status/1383545728488808453
I think it is more fruitful to realize that humans also make mistakes. It may also be also beneficial to compare Human + FSD driving vs. Human driving (instead of just FSD vs Human).
What about when Tesla's marketing says FSD can do[2] things it can't do?
[1] https://www.thedrive.com/tech/39647/tesla-admits-current-ful...
[2] https://old.reddit.com/r/TeslaLounge/comments/oey4g1/for_tho...
It doesn't mean they're directly lying, in the sense of "You can't reasonable get these numbers out of the dataset." But it probably means that there are some games being played such that the number you're getting isn't the number it's strongly implied (or, occasionally, outright stated) to be. Or, if it is, it doesn't mean what they'd like you to think it means.
So, if you go back a few years to the first waves of numbers Tesla trotted out to "prove" how amazing Autopilot was, you roughly had:
- The fatality rate per mile of all drivers, in all vehicles, in all conditions
being compared with:
- The fatality rate per mile of expensive luxury cars, driven by older, well off drivers, mostly on limited access divided highways, in "easy" driving conditions.
You can certainly do the math and get a comparison, but it's not a particularly reasonable comparison, because:
- In general, six figure luxury cars are really, really safe. They're big, heavy, loaded with airbags and safety features, and really don't kill people very often at all. You can go look at a breakdown of death by vehicle type and class, and it turns out that luxury cars are very much under-represented in fatalities.
- Expensive cars tend to be driven by older, more experienced drivers. There's a bathtub curve of fatality rates by age (https://aaafoundation.org/rates-motor-vehicle-crashes-injuri...) - younger drivers without much driving experience die more, and once you get to 70+, your fatality rate goes up, but there's a sharp drop going to the 30-39 group, continuing to decline up to the 60-69 group. Guess what age range is going to be mostly buying Teslas?
- Divided highways tend to be fairly safe places to drive - especially in lower speed situations where a lot of people are going to be using Autopilot. You're exceedingly unlikely to die doing 20mph in rush hour traffic. Even running at speed, limited access highways are fairly safe places.
- And Autopilot simply doesn't work in bad enough weather. Again, a lot of traffic deaths are in pretty sketchy conditions - rain, snow, sleet... things that older Autopilot, especially, simply won't work in.
So, yes, a straight up comparison of the two numbers ("fatalities per million miles in Teslas with Autopilot engaged" vs "fatalities per million miles of all vehicles in all conditions") is at least somewhat deceptive.
For one thing, Autopilot is more likely to be engaged for some types of driving than others. Comparing statistics with Autopilot engaged to the total of all miles driven is comparing apples and oranges.
Similarly, it's comparing statistics about the people who buy and drive Teslas with the entire driving public. You don't know how many accidents Tesla owners would have gotten into if they were in cars that didn't have self-driving features.
Finally, you can't draw any conclusions about whether lives were saved by just looking at the number of accidents and ignoring their severity. Hypothetically, if Autopilot caused fewer accidents at low speeds but more catastrophic collisions on highways, it could be easily be causing more deaths than it prevents.
In fact most things are better done by humans, because they understand context, which most AI/ML currently doesn't.
AI is great for tasks that can be done without much context and can be incredibly efficient and fast compared to manual work. Everything else, including "self-driving" currently needs supervision or produces terrible results.
The software will improve faster than humans will, as arguably, we've already hit our limit. Software doesn't get tired and it doesn't drive impaired. It makes mistakes, but so do we, and it's mistakes can be improved upon at scale. This is not blind faith in technology or software, but an acceptance of the limited capabilities of your median human (as well as the disadvantaged outliers).
[1] https://www.nytimes.com/2021/07/01/business/boryana-straubel...
This is not to say that software and hardware will not quickly overtake us, but while humans are on the road and in control, improvement will be possible.
You are right, the software should absolutely be held to a high standard (otherwise we're socializing losses and privatizing profits, also bad!), but please don't perpetuate a lie that existing drivers are held responsible to the degree they should for their mistakes (or even intentional acts) that cost lives. You'll find examples where someone is made an example of, but that is rare.
So disagree, please (checks and balances are important, as is the dialog we're having), but having seen what humans have to offer in this context, I will be the first, on my own time and resources, in front of regulators to argue for aggressive driver assist system development and implementation. Because the alternative is hot garbage we get by with because there are limited alternatives (ie no public transportation most everywhere and the inability to deploy it cost effectively), and we do a disservice to those who die or suffer because we could do better.
All depends on our faith in the system getting better of course. There's always going to be dissatisfying things surrounding it.
Gotta be up front about the state of the art though, can't let people sell a thing like it's the finished item.
Factoring in that bad drivers start being really negligent when there cars are “self-driving”, I'm not certain this will save more lives than the opportunity cost. But if it's mostly used by good drivers, and bad drivers get off the road, it could be huge.
I found lots of photos from the 50s trying to find more info
We will never get massive public transport, no matter how much money is spent. That is the harsh reality
Let's also assume our hypothetical public transport is built in San Francisco. San Francisco recently spent $1.5B on a 1.7 mile subway line. It will also take a decade from when work begun to its opening next year.
That works out to be a 9.7 mile subway that opens in 2032.
A million people are killed on the roads yearly world wide. How many deaths would you accept to reduce that to 10,000/year?
Zero.
As it stands, being able to blame Autopilot has a great chance of improving the system. What do you think happens when the Tesla Autopilot has one of its predictable jerks and maims a pedestrian? YouTube is full with documentation of just these kinda problems, and suing Tesla in civil court promises a lot more cash than some random deadbeat.
I'd like to see a Tesla on full autonomous autopilot take a drive in Calgary or Edmonton a few days after a heavy snow, when it's been mostly plowed but the lines are missing from the roads, all the lines at the side of the road are covered in snow, etc.
Often when driving like this, you see a glimpse of the middle-yellow line inside one of the wagon tracks, and you realize that you're actually driving in part of the other lane. Fortunately the opposing traffic's wagon tracks are also shifted so that they're driving partially on the shoulder.
A GPS guided car would make a mess of the wagon track pattern that all the humans decided to just go with.
A lot of autonomous car problems would go away if companies and governments went for enhanced roads instead of trying to create a generic solution on the first try. No need to detect traffic signs or lights if that information was made readily available by the road you are driving on.
The Tesla keeps seeing it as a traffic light then rejecting it as not a yellow light over and over in the UI. Just the right atmospheric conditions plus just the right angle probably don’t show up a lot in it’s sample sets.
GPS gives accurate timing and location which should make filtering out the moon straightforward, but it might just be overly cautious as a level 3 system.
Whatever stereo-depth perception of the camera they're using apparently can't tell the difference between where a traffic light should be and a celestial body. OTOH, I have mistaken a Burger King light-up sign for a full moon when it was coming up behind some trees, so I can't fault them too badly.
Watch a Tesla UI for a while and you will regularly show objects being reclassified. What’s breaking that loop is the fact the moons relative position is fixed so after rejecting it as not a traffic light the software then sees that same input and starts treating it like a new object after a while. I don’t know if the slowing behavior is related to the misclassified moon or a response to the model being confused.