Comment by 0x500x79 | Hacker News Reader

Comment by 0x500x79 | Hacker News Reader

I got curious. There is a tiny bit of rain in the video but according to tesla that should be supported:

"Many factors can impact the performance of Autopilot, causing the system to be unable to function as intended. These include, but are not limited to: poor visibility (due to heavy rain, snow, fog, etc.)"

But then it goes on to say: "right light (due to oncoming headlights, direct sunlight, etc.), mud, ice, snow, interference or obstruction by objects mounted onto the vehicle (such as a bike rack), obstruction caused by applying excessive paint or adhesive products (such as wraps, stickers, rubber coating, etc.) onto the vehicle; narrow, high curvature or winding roads, a damaged or misaligned bumper and extremely hot or cold temperatures."

What? This is an amazing set of limitations that are normal occurrences on the roads I drive every day. These do not seem like edge cases.

I hate the marketing from Tesla on this feature. When it is mentioned in press it makes it sound like the self-driving/autopilot is nearly done when in fact there is a ton of work that needs to be done here and a lot of people question whether their current approach can even work. I am by no means a self-driving expert, but this terrifies me.

From the results in the video it is dangerous to have this on the road. I do not want my family anywhere near a car that is operating in autopilot mode.

It's the thing that nobody really talks about and conveniently ignores because if you do it means Tesla (and other AV companies) will go out of business.

From my experience - if you don't have well maintained roads you're going to have problems with FSD.

Raining? Water running over the camera/sensors will warp the image. Snowing? Can't see the road lines.

Tesla can't even guarantee that their cars won't barrel directly into a fire truck that's stopped on a freeway trying to address an existing accident.

In Canada, there's a federal mandate that forbids the use of oil-based roadmarkers - provinces instead use water-based roadpaint... you can literally see each year how that works out. Worn down/invisible road lines make it difficult for even experienced drivers to work out how many lanes a road has - good luck getting a computer to work out what lane it's in without any exterior help.

Without some kind of guidance inside the road itself - RFID/Wireless tracking "beacons" of sorts to help keep the vehicles in their lanes - we're not going to see true FSD any time soon globally - probably just in nice climate countries/states like California, Texas, etc where the weather 99% of the time never changes from "clear and sunny".

There's just too many variables for FSD to account for - especially with Tesla choosing to not use other tech like LiDAR, like others have mentioned.

FSD isn't going to be around in any major capacity globally for at least another 15-20 years at least. Too many factors involved in making sure your multi-ton autonomous spontaneous death machine picks the right option when it sees the equivalent of the trolley problem with potentially only a few seconds to make the decisions.

> Tesla can't even guarantee that their cars won't barrel directly into a fire truck that's stopped on a freeway trying to address an existing accident.

Isn’t this a design limitation of radar ACC systems, not FSD?

Not that I’m saying FSD will reliably avoid stationary objects (I have no idea), but afaik all such collisions so far were under radar Autopilot, which by design cannot see stationary objects.

Yeah, I might not be remembering the story correctly. It might have been collision avoidance rather than the FSD portion of their firmware.

I still don't think that FSD is really for prime-time in any capacity though. Seems to make far _far_ too many mistakes that users need to correct. Does that sound like Full Self Driving to anybody? :P

Maybe Tesla radar can't see any stationary objects, but others use radar for collision avoidance / auto braking and seem to do so also for stationary objects. It's harder to do from a signal processing view, but definitely not impossible.

None of those limitations should have mattered in this case, since they all relate to the sensors. We can see the Tesla's view of the world on screen. It knows there are double yellow lines. It knows there is a car in the oncoming lane. It just decides to drive diagonally into oncoming traffic anyway for some reason. There's no way you can blame this on sensor problems. It's just a plain old software bug.

I think this is also a perception failure. If you closely watch the video, the detected car appears on the visualization way too late after the Tesla has already swerved to the oncoming lane.

A good enough sensor stack (read Lidar) would have detected a solid obstacle at X distance (real measurement) way earlier than a camera making sense of the pixels, classifying it as an object and then providing an ML estimated distance (not real measurement) to the object. All you need in this scenario is to be able to tell there is an obstacle without having to even know it's a car, which Lidar is very good at doing.

Hmm, the view of the screen isn't clear enough to tell when exactly the car appears. An eerily familiar problem, lol. But that's kind of beside the point. Here's a screengrab of slightly sooner, at 8:05.

https://i.imgur.com/Hj6glgu.png

The car made a plan to cut across the yellows and then continue straight down the opposing lane. It shouldn't do that whether or not it sees traffic up ahead.

https://imgur.com/a/y1MDoUn

Here's the screen grab of when it decides to cut across the yellows (see the path plotted). The car is clearly visible in the oncoming lane with headlights on. There's also another car closer in the farther left lane. Neither of them is detected in the visualization. This is where a system with more robust sensors would detect objects way earlier and would never decide to cross lanes, whether it's double yellow or not.

https://imgur.com/a/o7hV45F

Here's after it has crossed the yellow lines. The car is again clearly visible, but no detection yet. Clear perception failure.

Interesting how the oncoming car appears to have one brighter headlight than the other. I wonder if that's what confused it...

Following that thought... confused it into thinking what? (Allowing for anthropomorphic use of 'thought'.)

Really, the key here is that it's on-screen model shows the yellow lines. It must have become terminally confused to plot that course. Looks like it threw away the model and the rules and just decided to end it all.

Headline: "Tesla computer, in fit of depression, commits murder/suicide". (With a number of !s appropriate to the publishing venue.)

What's the name for the slash marks sometimes found on suicide victims, where they tried to cut their own throat or wrists several times before they succeed? Hesitation marks? Maybe that's what we're seeing here. /facetious

I agree there were perception failures, but that's besides the point. The car made a plan to drive on the wrong side of the road. It shouldn't ever do that, no matter what it perceives. It's illegal whether or not there's oncoming traffic. You can't blame sensors for those kinds of logic errors.

On the plus side, the computer did say "I'm sorry" after almost killing the driver. So I guess that's something.

And right after that:

"The list above does not represent an exhaustive list of situations that may interfere with proper operation of Autopilot components. Never depend on these components to keep you safe."

Also, what are "extremely hot or cold temperatures"?

>I hate the marketing from Tesla on this feature.

There's a pattern in software marketing you eventually learn to recognize. Almost all of what Tesla does is just repeating this pattern.

It basically only works in California's wide, straight and sunny streets. Everything else is a death wish

"winding roads" is a peculiar limitation. That would exclude a good chunk of the United States. Older roads that aren't high in traffic today often just follow whatever random path the wagons used to take.

If that's really Tesla's position, FSD is absolutely unusable in Germany. Roads are being build not straight on purpose to counter fatigue in drivers.

You are being reasonable based on the data you have, but just like teenage girls on instagram your reality is being distorted by the data you are fed. It is well established that teenage girl mental health is being hurt by that information being shown to them over social media, specifically information that makes them feel less attractive or less self confident.

Similarly, if you watch a subset of car videos, specifically "Tesla FSD Failure" videos, you will believe that FSD is a huge problem. If instead you had watched videos of people leaving bars and manually driving cars after 8:00PM and a subset of FSD videos driving perfectly, you would come to the conclusion that you want your family no where near bars instead of FSD.

The NTSB is our best hope at actually making driving safer and not just knee-jerk reacting to a few dozen videos.

you shouldn't be allowed to operate a vehicle while intoxicated regardless of the technology involved.

I don't think that was the point they were making.

If you watch videos of drunk non-Tesla drivers you get the impression manual driving is unsafe.

But on a side node the long term aim is hopefully that even drunk / sleeping / 10-year old people can use self driving cars.

The original post is "fine", although a bit antagonistic. The point still stands though.

I do not think that this feature should be on the road and it's not "that the car is drunk" it's that the Autopilot feature is fundamentally is mis-designed. Take a look at any thread related to Tesla Autopilot and you have experts calling out that the lack of Radar/Lidar is absolutely reckless. This is a clear reason why: the cameras ability to discern objects is limited and can cause erratic behavior under normal circumstances.

The cost/part cutting option of removing these sensors (or not using them) has time and time again shown that it can act erratically. I don't want to have drunk drivers on the road and I don't want this Autopilot on the road. Both are bad options, but the way that this is marketed to the every-day person makes it seem like it is ready to ship and you can "be drunk and Autopilot will take the wheel". We are quite far from that. FSD needs to exceed the capabilities of a human or prove that it has dramatically less deaths/accidents per mile than human drivers.

I work in software, I work with ML algorithms. I don't trust either with mine or my families life right now. I know that there are life-critical software deployments, but who is regulating Tesla Autopilot right now and are they doing enough? The NTSB is trying... we will see how it works out.

But if the likelihood of FSD causing an accident is N%, but the likelihood of a human error (alcohol or otherwise) is M%, I would always choose the better bet.

I realize the error situations may differ drastically but if the final numbers are in favor of the computer, it's worth betting on. IMO.

Sure, at the moment that is by no means clear cut yet. The statistics on self-driving are if anything: "computer with human watching over it in near perfect conditions" has lower probability of accident than humans in all conditions. Anyone who cites the statistics in any other way is deliberately misleading.

This brings up a second question, if a human driver would do something like in this video and cause an accident, there would very likely be criminal charges. So if an self-driving car does it, should there be criminal charges against the the engineers, the CEO?

Bear with me here. You find random Hacker News comments convincing. But what if you look at the evidence instead of commenters?

Let me save you some time by linking you to an engineering talk by the head of AI at Tesla. I'll link you directly to a time in the video where he directly contradicts the witness of your expert hacker comments by showing the empirically results of sensor fusion side by side with the empirical results of vision only.

https://youtu.be/a510m7s_SVI?t=1400

I don't understand this argument. So because people drive drunk we should accept self driving to utterly fail? Because FSD does better than a drunk driver it's safe? Why can't both be true, that drunk driving and FSD is dangerous. Moreover why are self driving cars the only solution to drunk driving? It's not like the police doesn't know where the bars are they could easily pull all these people out in a few sting operations, make penalties extremely ainfjl and things stop very quickly.

There are 30,000+ car accident related deaths in the US alone every year. There are 365 days in a year. That works out to an opportunity for roughly ninety videos of fatal car accidents every day. Watch the ninety videos of people dying before watching the one video in which no one dies and maybe you will be able to understand the argument better. The actual raw data - the safety statistics - are very clear. Human + AI is safer than human alone and Tesla's cars have the lowest probability of injury given an accident of any of the cars on our roads. The statistics are so opposite your thesis. The reason you have your thesis is because you have been misinformed by the selection bias.

Reply on news.ycombinator.com