Tesla FSD Beta Barely Avoids Crashing into Pole
twitter.com
twitter.com
Here's the full video: https://youtu.be/bbyNg9kYEq4
The guys seem to be Tesla fans, despite needing to intervene multiple times to avoid crashes. Also, I noticed that it displays concrete pillars as cones.
Also, is it really just a LIDAR issue? LIDAR would help with the pole, but it wouldn't help with the car wanting to turn onto railroad tracks, changing its mind, and then swerving back for the railroad tracks.
It remains unbelievable to me that this is allowed on the roads. I actively avoid Tesla's on the road.
Mission #0 for self driving is to never collide head on with something. Radar car systems that automatically apply braking from 20 years ago could achieve this. IMHO, forward facing radar running without AI and straight up deterministic algorithms should be able to take control away from the ML systems when a collision in imminent.
Really, this is fixable. The car can see streetlights. I mean, duh. This wasn't an out-of-regime situation. There's a bug, mostly likely in the contrast control layer (the lamp was in a shadow). Bugs can be fixed, that's what beta is for.
This is going to be a popular genre ("Did you see what that Tesla did?") for a while as the new capabilities appear in public. And then they'll be fixed, and we'll go back to arguing about other stuff.
It seems nuts to me that Tesla can’t do better than a ~$1,500 quadcopter.
That's probably because you've mischaracterized the issue. It's a streetlight. Obviously Teslas see streetlights. Go for a ride in a friends car (not even one with FSD Beta) and look at the screen. You'll see streetlights rendered by the hundreds. It's basically perfect. It failed here, not because it couldn't see "a pole" but because of a much more specific bug. Specific bugs can, almost always, be fixed.
Here's the latest, even when it finally succeeds, it is not very convincing or consistent.
Or do you think that the progress we're seeing currently will just stop? You think that video is as well as the car will ever handle that particular situation to your satisfaction and no further progress is possible?
Most software has to deal with a limited input domain, and when mistakes are made, they are tolerable. And even in the case of software in limited domains, critical bugs remain in commercial software that have been around for decades, with new bugs cropping up as the software is maintained.
Now look at FSD: What is the input domain? The entire real world and the incredible diversity of variety of it. And what is the repercussions of a bug? Death or injury.
So like I said, these FSD systems should be designed with a kind of "kernel" vs "user" space paradigm. The "kernel" has a mission to absolutely prevent "user" space from crashing the system. In my view, that would mean a backup sensor suite and software design whose highest priority is to avoid collision, rather than driving or navigating. All of Tesla's ML systems should be in the "user" space. They view the world, make predictions to navigate, but any control inputs they yield should always be overridden by the "kernel" if a collision is pending.
The vision has to be good enough to handle this type of situations with acceptable error rate.
Also, sensor fusion is difficult - LIDAR does not see colors, Radar is just weird.
Adding more complexity will eliminate some bugs, and result in new bugs.
That's non-falsifiable. You can end a discussion that way, but not win an argument. Let's just leave it at "FSD beta appears to be improving rapidly" (c.f. the public history of this stuff since the first public beta users received the build in March) as my response.
I don’t think I have.
The OA feature on my quadcopter positively prevents you from flying into obstacles, even intentionally. It’s not perfect - power lines, wires, and bare branches aren’t seen 100% of the time - but I’ve never had it trigger a false positive.
Why wouldn’t there be a similar system at play here, limiting the actions of the FSD?
Tesla's FSD/Enhanced AP cannot be safer than a human driver + passive aids like LKA and AEB. The statistics that constantly get paraded about AP are vs all cars, not just cars with passive aids.
It's simple enough:
If FSD would catch a hazard with the sensor suite, so would the passive aids.
But if the sensor suite would miss a hazard, it's not an issue unless the human does too when it comes to passive aids.
If the human misses it and the sensor suite misses it, so will FSD.
The problem is this: With FSD the car will actively do the wrong thing in response to the missed hazard instead of doing nothing. Now the human has to correct after the wrong action has already been taken.
-
So if you visualize a venn diagram of accident conditions, the ones prevented by FSD will fit inside the ones prevented by passive aids, since FSD is actively creating accidents where they couldn't exist before.
I mean, you're right that if you take a putative autonomy solution and add another redudant sensor system (not even a human) that you'll catch situations the first wouldn't. But if that system is then allowed to demand control, then you open the possibility of mistakes in the "supervisor layer". So you're still limited by the best available quality of one system.
Specific example: go watch highway crash videos on youtube for a while. A very common situation is a "late stop" situation on the highway where a car fails to see the one its following stop in time. Most drivers just slam on the brake and hit the car ahead, but a significant fraction swerve instead. And those swerves cause rollovers and jacknifes and all sorts of worse accidents.
So... how does your hypothetical "human driver + passive aids" prevent the driver from incorrectly swerving at high speeds?
> You're assuming the human always makes the right decision, which is wrong by construction in this particular situation as the problem being solved is the fact that drivers mess up.
I don't really get what you mean here. "which is wrong by construction in this particular situation as the problem being solved is the fact that drivers mess up." I read that like 5 times now and it doesn't parse.
I'm not assuming the human is right. I'm saying if the human would do the wrong thing, the passive systems take over and do the right thing. If the passive systems can't figure out what the right thing is, FSD can't either.
They're all using the same source of truth, Tesla isn't shipping an AEB implementation that performs worse than FSD...
-
> I mean, you're right that if you take a putative autonomy solution and add another redudant sensor system (not even a human) that you'll catch situations the first wouldn't. But if that system is then allowed to demand control, then you open the possibility of mistakes in the "supervisor layer". So you're still limited by the best available quality of one system.
But you seem to have gotten the point and just not applied the right direction. Imagine the supervisor layer is FSD. You're still limited by the best available quality of the system... the human.
-
> Specific example: go watch highway crash videos on youtube for a while. A very common situation is a "late stop" situation on the highway where a car fails to see the one its following stop in time. Most drivers just slam on the brake and hit the car ahead, but a significant fraction swerve instead. And those swerves cause rollovers and jacknifes and all sorts of worse accidents.
This is a pretty dense example... you're talking about several different scenarios that each deserve their own points. But to go with just the part you focused on at the end:
> And those swerves cause rollovers and jacknifes and all sorts of worse accidents.
>So... how does your hypothetical "human driver + passive aids" prevent the driver from incorrectly swerving at high speeds?
Passive systems already exist in almost every car on the road to make those swerves safer and rollovers less likely. That's Electronic Stability Control. Thousands of times a second your car is monitoring for the conditions that would cause a rollover.
The best cars will actively straighten the car after the swerve to allow the driver full control. It's smart enough to sacrifice responsiveness to give the driver a predictable result.
Watch how the car ends up sliding thus "failing", but does so perfectly facing the original direction of travel, almost as if the car was capable of driving sideways. Done with incredibly millisecond precision in the application of the brakes, something no human could ever replicate: https://www.carthrottle.com/post/porsche-has-issued-a-respon...
ESC systems are already designed to work in harmony with AEB: https://www.zf.com/products/en/cars/products_31616.html
So now the ESC system can be designed to operate in a manner that does even better when short stops occur (the "ESC high dynamic" mode is designed for the AEB's sudden application and the resulting driver inputs that will happen after)
-
Seriously, 99% of the hype about FSD seems to be people who don't realize what how active "passive" systems are when they are activated.
That is the key difference, only working when the human is already in full control, so that if the passive system is wrong, the human is already providing the correct input.
It's like trying to correct FSD running towards a pole, vs just ignoring an incorrect LKA system that tries to guide you towards the pole.
With one you're already headed for a pole. With the other, you were never headed for the pole, the only way you hit it is deciding "yes I would like to hit a pole". Worst case for both is the same, but average case is better when you're ignoring something, vs correcting for a mistake already made.
Now you've lost me. I thought you were constructing your argument as "human+machine > machine_only". Which is wrong, but I thought I understood it. Now you seem to have gone in circles.
How is a passive system deciding when to "take over" and do the "right thing" any different than a full autonomy solution? What if the passive system has bugs too and does the "wrong thing" instead of the "right thing" (or fails to do the "right thing", etc...)
You don't get to assert that your magic system will always be better than "FSD" by taking its perfection as an axiom!
You should have stopped replying and learned how the systems in question work, they are not "magic".
-
FSD is not "machine_only"
FSD is a system that supposedly will give way to a "machine_only" solution, but currently is a human+machine system. You literally just watched it in action if you watched the video above.
Maybe not realizing this is the crux of your confusion.
> But if the sensor suite would miss a hazard, it's not an issue unless the human does too when it comes to passive aids.
That certainly sounds like you're saying that FSD is "machine only", since you're ignoring the ability of the human driver to correct it. If that's not what you meant, maybe you can start over from scratch and explain your point?
-
Nothing about what you quoted implies FSD is machine only.
And I mean it's not even for me to define if FSD is machine only, FSD has never been machine only, that's why it's even allowed to operate as it does on public roads.
-
Now, my final attempt to break this down in yet another way...
A driver using LKA needs to actively be applying the correct input since it only works when the car is already approaching a lane marker.
A driver using FSD must be doing the exact opposite. While they must be paying attention and ready to provide input they cannot be actively applying the correct input without causing FSD to disengage.
So most they can do is give the systems indications that they are ready for after things are off the happy path.
-
Both LKA and FSD can fail for inattentive drivers and both can save them, but because of the difference in input, only FSD can harm a driver who was otherwise aware and capable of providing the correct input but not allowed to until things started going wrong lest the feature disable itself.
This is simply not the case. If you don't like what FSD is doing you disengage. The very linked video we're discussing is an existence proof!
Is your whole point a psychological one? You think FSD drivers are less likely to correct the machine than drivers with assistive technology? That seems like something you need to argue with evidence and not principle.
Are they? I certainly haven't seen numbers. Tesla's own numbers seem to imply the opposite: Teslas also have assistive features, but they still get in fewer accidents with AP engaged.
> This is simply not the case. If you don't like what FSD is doing you disengage. The very linked video we're discussing is an existence proof!
How do you use FSD and actively steer. When you actively steer FSD will disengage. That's literally what my comment says there. You cannot provide correct input while FSD is active
So you're going to have a worse reaction time, and be starting from a worse situation than someone who was already hands on wheel driving.
And the passive aids still help the person who is hands on wheel driving in the situations using the exact same sensors as FSD where they're distracted so you're not giving up safety.
> Is your whole point a psychological one? You think FSD drivers are less likely to correct the machine than drivers with assistive technology? That seems like something you need to argue with evidence and not principle.
I've argued it with simple logic you could have easily worked through if you actually understood the thing you were talking about for 4? 5? comments now?
The hilarious thing is how you clearly haven't tried looking to find the multiple studies that come up with the same result: https://electrek.co/2021/09/13/tesla-autopilot-decreases-dri...
> Tesla's own numbers seem to imply the opposite: Teslas also have assistive features, but they still get in fewer accidents with AP engaged.
You're saying Tesla's numbers enforce the narrative they're pushing? Say it ain't so.
First off I've never seen these numbers for FSD, so I'd love to see them.
Second, comparing Teslas to Teslas is such an easy out for them, why wouldn't they?!
I mean, Tesla has largely lagged behind the rest of the industry in passive assistance. They're focusing on more active features, and the result is things like getting basic LKA without human intervention in 2019, almost 5 years after the base model Corolla did: https://www.tesla.com/blog/more-advanced-safety-tesla-owners
-
But the fact only counting vs Teslas is such a huge confounding factor... it's laughable someone would actually pass that as acceptable.
Just the fact that AP being on means you can never go more than 5 mph over the posted speed limit, hello? People with AP have to disengage it to drive recklessly, so of course if you limit yourself to Teslas they'll be much safer.
Then AP is mostly designed for the types of roads that have the lowest accident rates.
And the inverse AP straight up doesn't work on the roads with the highest accident rates, only FSD does and we conveniently won't get that broken out for a loooong time...
It's embarrassing how many obvious holes there are here that show they're
-
I mean don't even take my word for all this: https://www.businessinsider.com/tesla-crash-elon-musk-autopi...
No one has shown the only useful metric yet. Newer cars with features like AEB and LKA vs cars with FSD.
Tesla is happy to keep playing the media circus with things like comparing accident rates with _all_ cars on the road... everything from a 20 year old compact with the crash safety of a paper bag to a lifted pickup truck with a center of gravity somewhere above the average car.
In the meantime we'll continue to watch videos like the one above, while companies ship cars that have safety features for when things go wrong instead of claiming to be "Full Self Driving" then ask you to be ready to take over at a moments notice.
So... your point is that the mere fact of "actively steering" causes a quantifiable difference in attentiveness not seen in data, but that you feel is so obvious as to be a first principles argument that doesn't need any?
I disagree. And my evidence is that drivers watching existing Tesla autopilot are seeing the same interface you're decrying, and are getting in fewer accidents than people using the Tesla-provided assistive feature but not using autopilot. (c.f. https://www.tesla.com/VehicleSafetyReport) Also, that an attentive driver not "actively steering" is clearly sufficient in practice, as this video proves.
Your turn. You claim this isn't as safe. Where's that been measured?
I'm done repeating myself, the answer to what you posited is in the comment you replied to.
And I remain confused. You haven't presented anything here that looks like a measurment or number. I mean, if passive assistive technology (deployed in real cars) is clearly safer than drive-on-its-own automation (also deployed in real cars, c.f. Tesla AP), then that should be measurable. And in fact at least some measurements have been done (by Tesla, of its own fleet) and show the opposite.
All you have to do is show me the data that shows this is incorrect. But... you didn't. You're just citing an "answer". Answers aren't enough to rebut data.
It sounds like a small difference, but it completely upends 99% of the assumptions you can make about its performance.
-
For example you can't response to passive aids, you can't use FSD.
Look at the video we just watched, would a tired or distracted driver have been able to make that correction?
-
Passive aids even handle tired drivers _better_ than FSD.
Toyota's aids will warn the driver they are drowsy if the passive aids are intervening too often. They've done this for years now, even in their most basic cars.
FSD is assuming computer control with human backup, so the human backup isn't being engaged enough to make calls that well. At best it can use proxies like wheel jiggling and gaze.
But it's reasonably clear that this system, even in this state, isn't causing accidents. I think an open beta like this is absolutely appropriate. Lord knows I'm enjoying watching it (but not using it yet, didn't make the Safety Score cut).
For me it also shows how much in the ball you need to be while supposedly relaxing on FSD. This really was a split-second decision, out of nowhere, with no warning.
And, presumably, since in FSB you’re supposed to be alert all the time, insurance will point the finger at you for driving into a lamppost. Not Elon’s fault at all.
Anyone supervising it for longer than 15 min will realize that Tesla means it and that it is not some lawyer speak.
The FSD Beta will become dangerous when it will start driving flawlessly for 50h, enough to make people complacent, and crash into a pedestrian on 51h.
The whole video on YouTube is a disaster. The car turns left across facing traffic on a red left arrow. Easiest machine-vision problem you can imagine: chroma-keyed directional signal.
If you take control, Autopilot is off so you're the one who crashed instead.
I kinda figured something like that from the context, but not hanging on the days and lives of our Tesla future, this is the first time I heard this particular acronym.
Can we (HN community) please recommit to spell things out on first use unless the abbreviation is long and established and unambiguous?
That said, I see sign poles flattened all the time. Paint and tire marks all over barriers, walls, and islands. People crash all the time. The old anecdote game isn't really useful, automatic driving could be 100x better than human drivers in all measurable metrics and you'd still be able to upload dashcam videos of it fucking up.
Yes. CA DMV accident and disengagement data for self driving systems using Lidar show they are indeed better (and by miles).
Note that Tesla refuses to share FSD beta data because they classify it as a level 2 system. I'm willing to bet their disengagement rate is in the single digits. For comparison, Waymo and Cruise both had around 30,000 miles per disengagement in 2020 according to CA DMV records.
This seems to be a self reporting scheme for companies testing self driving cars in the state. I could find little or nothing about the details of the testing being done or methodology used by each company.
> Note that Tesla refuses to share FSD beta data because they classify it as a level 2 system. I'm willing to bet their disengagement rate is in the single digits.
Hypothetically, could company A use company employees as human drivers driven specific routes chosen by the company, while company B used volunteers from the public in their own cars driving their own routes, and both data sets would go into the same CA DMV data?
Could company A have any incentive to improve miles per disengagement to the point where they might intensively train their drivers, direct them about certain situations which don't require disengagement, etc., choose to drive on routes known to be less difficult? And could the drivers employed to do so much test driving rapidly learn more about the behavior and know when they can avoid disengagement?
If so, then I'm a bit skeptical about the objectivity of that data.
Given Tesla FSD numbers are not included, are you saying that data is showing lidar systems are better by showing numbers for e.g., Waymo and Cruise vs other non-Tesla companies which don't use lidar? Which companies are those? I'd like to understand how you're using this CA DMV data.
It is a self reporting scheme. Waymo publishes their methodology at [1], Aurora at [2]. I'm not aware who else publishes their methodology, so it's up to you how much you trust them.
> Hypothetically, could company A use company employees as human drivers driven specific routes chosen by the company, while company B used volunteers from the public in their own cars driving their own routes, and both data sets would go into the same CA DMV data?
Yes, why shouldn't it go to the same dataset? Company A tests in an area defined by their Operational Design Domain (ODD), company B's ODD is simply everywhere in CA (that's what they've committed to).
> Could company A have any incentive to improve miles per disengagement to the point where they might intensively train their drivers, direct them about certain situations which don't require disengagement, etc., choose to drive on routes known to be less difficult? And could the drivers employed to do so much test driving rapidly learn more about the behavior and know when they can avoid disengagement?
Sure, they can drive only in routes that are less difficult. But they have a product to develop (a public robotaxi service) and areas to expand. So they don't really have an incentive to exclusively work towards artificially reducing disengagement rate.
> Given Tesla FSD numbers are not included, are you saying that data is showing lidar systems are better by showing numbers for e.g., Waymo and Cruise vs other non-Tesla companies which don't use lidar? Which companies are those? I'd like to understand how you're using this CA DMV data.
Nope, I indeed mean they are better than Tesla. There are no other (aspiring or current) L4+ systems that don't use lidar. When Tesla used to report disengagement rate in CA, they were still worse than their competitors. That and the tons of YouTube videos showing FSD being unable to handle basic scenarios tells us their current disengagement rate is far worse than competitors.
What they should be really doing is start being transparent and report safety data like everyone else.
I didn't say it should or shouldn't, I was just asking about the data you are using.
> Sure, they can drive only in routes that are less difficult. But they have a product to develop (a public robotaxi service) and areas to expand. So they don't really have an incentive to exclusively work towards artificially reducing disengagement rate.
Oh they absolutely do if people buying their shares, their services, employing them, or writing legislation, or just general public attitudes are influenced by this data.
The marginal cost of cars is so low given the costs involved that it would be trivial to add easy miles while still doing difficult routes to develop the product. Including doing most of the difficult work in other states.
> Nope, I indeed mean they are better than Tesla. There are no other (aspiring or current) L4+ systems that don't use lidar. When Tesla used to report disengagement rate in CA, they were still worse than their competitors. That and the tons of YouTube videos showing FSD being unable to handle basic scenarios tells us their current disengagement rate is far worse than competitors.
Okay, so not the objective data I wanted but thanks for helping.
> What they should be really doing is start being transparent and report safety data like everyone else.
I don't think this is really a matter of transparency given all the problems with the data and tests we've just identified. When the products and are made available to the public that would be a different matter.
None of this matters if they don't show product development. They have a market to serve and profitability to think of. There's only so much PR you gain out of artificially reducing disengagement rate. You can't seriously believe this is a real strategy employed by a dozen SDCs.
> I don't think this is really a matter of transparency given all the problems with the data and tests we've just identified. When the products and are made available to the public that would be a different matter.
Even if you think self reporting data is a problem, the solution is not to encourage companies like Tesla to continue to be opaque and not comply with rules. Since Tesla FSD is already in the public's hands (with more weekly rollouts planned), it is indeed a matter of transparency.
If so, I've seen quite a few of those for the Tesla FSD beta, but none for any other cars. Not that I try to find them, the Tesla hype must just make those come up more. Do you have any of them with other cars showing them go through these same routes without problems, that FSD struggles with? That might be fun to watch, even if it's not significant actual data.
In one sense yes: accurate ranging is an inherent capability of a lidar system that a passive optical system needs additional algorithms to achieve (e.g., simultaneous localization and mapping (SLAM)).
In an active system like lidar you know when you’ve transmitted energy and you can measure the delay until you receive an attenuated version of it scattered from objects in the environment. And you know the velocity of the energy so you can solve for distance.
The big fucking difference is that the latter was a system installed into all of them! The failure mode was systematic! A different car with the same software facing a similar situation would go straight for the pole again!
(The best theory I've seen presented for this particular bug, FWIW, is that it was the shadows and training data at fault. Teslas can, obviously, recognize street lights. But this one evaded detection at a distance because it was inside the dark shadow and presumably the contract detection failed to resolve it. And once it was in the near field, the recognizers don't have as much training data and failed to see it.)
This was probably true in 2016, but not anymore. Waymo’s 5th gen lidar and Argo AI’s lidar all show massive performance and resolution improvements. They have been game changers. Anyone not using lidar in 2021 is just handicapping themselves, as we are seeing with Tesla FSD performance.
As far as I know the specs for Waymo's 5th generation lidar are not publicly available. Can you link to them?
How much work did you do with Waymo's 5th generation lidar? Or any lidar for that matter?
Do you have any qualifications to claim that a lidar that isn't even available to you is capable of doing a very specific task?
If you want to hear from an expert (which I am not), here is Brad Templeton talking about Waymo and Argo's lidar improvements [3].
[1] https://www.youtube.com/watch?v=COgEQuqTAug&t=11601s
[2] https://www.youtube.com/watch?v=I_tYxMh3ddA
[3] https://www.forbes.com/sites/bradtempleton/2021/10/11/new-li...
The best analysis I've seen is that this is an imaging problem. The lamp in the shadow didn't have enough contrast to show up and be presented to the ML recognizers (and, probably, that when it did it was too late to see the object in totality and there wasn't a recognizer for "vertical pole in the very near field").
[1] https://www.google.com/maps/@37.3347669,-121.8882277,3a,75y,...