Crashes caused by Tesla Autopilot are piling up, and there are consequences | Hacker News Reader

Crashes caused by Tesla Autopilot are piling up, and there are consequences

skynettoday.com

64 pointsandreyk24 comments

Since it's made explicit to anyone opting into the FSD Beta program that they must pay attention and keep their hands on the wheel at all times, this is a dumb FUD article. Drivers choose to accept full responsibility and opt-in to beta testing.

In one of the fatal crashes mentioned, the driver wasn't even in the driver's seat. Come on.

This is a case of "robot scary." I'd like to see the accident rate compared to human drivers in identical situations, broken down by age range. I'd wager you'd still see a massive reduction.

It's not useful to spotlight and misrepresent a set of attention-grabbing incidents, presented free of either statistical or situational context.

Absolutely not surprised that they don't handle cases like emergency vehicles in the road. These cases can be arbitrarily different to a normal road situation. No amount of AI trained on any amount of data can prepare a system to deal with all the possibilities (maybe you can get better on average than a human, or maybe not, but people will still be killed where a human driver would avert an accident). It's hard enough for a human driver to analyse the situation when approaching an accident or construction zone and realise that the shape in the road is a piece of shed load or a human or what.

Are your saying no amount of AI training is enough to recognize a car in front of you?

Yes, because depending on the situation, the car could be upside down, sideways, with half a side torn off, fire, smoke, flashing lights, flares, temporary barriers, humans both standing and lying, with bits of car everywhere. In the rain or snow, at dusk or with low sun angle. Or maybe the car is actually fine but just in an unusual place, but is covered in decals, high-contrast markings and equipment (eg a police car) that confuse the model just enough astray to enter a dangerous situation.

You can't train an AI for every combination of that. Hell, sometimes humans can't assemble a mental model of the situation in time to take safe action.

How many such crashes happen in general? Is it a common thing in the USA? Are humans in any other car less likely to crash into standing cars on the Highway? Any experts here?

When multiple crashes happen then a common reason is always tried to be found. Same car? Same system engaged? Same turn? Same section of road etc. and then that common reason is attempted to be fixed. Like if multiple accidents happen on same section of road, then it gets noticed and gets fixed (or someone sues the city to get it fixed). Here Tesla autopilot is the common denominator, so it needs to be fixed. Since Tesla isn’t willing to fix it or even accept that it’s dangerous, regulators need to investigate to determine if the system is indeed dangerous and if so, ask/force Tesla to fix it

Maybe camera only was a mistake? Researchers in this space have been saying that since DARPA Urban challenge in 2007, but Elon thinks he knows best.

What set of researchers? All researchers? "Many people are saying" energy.

Camera is just fine IF you can produce depth data out of it (/1). Evidently Tesla cant.

/1 You might want to go for Stereo setup to be on a safe side, but on the pillars not 2cm apart. Tesla didnt even opt for that. First version using Mobileye had something hilariously bad like single 30fps 640x480 monochrome/red(limited ability to recognize red/green lights) front facing camera.

Tesla can produce depth data from vision. A few years ago they couldn’t but they have been since early 2021 I think.

If they could they wouldnt be driving into big stationary objects.

It's the driver who is driving the car, not Tesla.

Obligatory “Tesla != Elon”. You’re acting as if “researchers in this space” and Tesla are separate entities, and they’re not. Tesla employs people with very good research track records, notably Andrej Karpathy, who is Director of AI and has a decade of ML/CV research under his belt and was a founding employee at OpenAI. Andrej has enough clout independent of his work at Tesla that if he thought Tesla’s approach was stupid, he could just leave and get a similar job at a company he thought had a better approach. Not saying he won’t do this eventually, but he’s been working on FSD since 2017 and hasn’t jumped ship yet, so presumably he likes the direction they’re going in.

I think they are making a long-term bet here, likely in their ML models as they acquire millions of miles of driving data. The problem is an ethical one, as they are wagering today's safety, and the very real people and property that stand to be harmed.

> The problem is an ethical one

No, it's a reputational problem now. Musk and Tesla have pointlessly backed themselves into an anti-lidar corner. They're going to look foolish when they finally do add lidar to the cars.

Lidar costs are reducing and it is making its way into more production cars:

https://www.bloomberg.com/news/articles/2022-02-08/elon-musk...

They are not mutually exclusive. A company can act (or choose not) based on affects on reputation, but this doesn't eliminate the ethical component of those decisions. You needn't consider the ethics of a decision for it to have ethical implications.

Vision seems to be working pretty well for them. I don’t see them changing course. FSD keeps improving.

Sure does: https://www.youtube.com/watch?v=bbyNg9kYEq4&t=340s

No problems: https://electrek.co/2022/02/04/tesla-full-self-driving-beta-...

It's working great: https://www.thedrive.com/news/44216/this-tesla-self-driving-...

The machine learning data also would be more valuable if it came with depth data. They could always produce future products which are camera only if their models would be good enough. It is probably just a means to get extra funding without having to spend much more on hardware.

They trained some of their initial depth vision networks with LiDAR.

This has the same problem as hypothetical sales.

You can justify anything if you're willing to accept it into your moral/ethical calculus.

I prefer less rationalizatiom and Farquading of others.

Vision only is probably better than mixing different sensors. Radar brings a whole other set of issues. BMW and other driver assist systems with radar all have a class of failures related to the detection of stopped objects when traveling at high speed. The radar is giving a confusing response and the vision disagrees. So which one to trust is hard.

Tesla’s vision is just a better approach. It’s how humans drive. FSD is using all vision.

The argument “more sensor data = worse” is a pretty hard one to make, especially in the AI world. If it were the case radar is bad at detecting stopped objects at high speed and vision is better there, wouldn’t the model learn this?

Yes, humans drive with just two eyes. But we don’t have LiDAR. If humans had LiDAR, you can bet we would use those when driving too. If Tesla had had LiDAR, it seems reasonable to believe at least one of the autopilot involved deaths would have not occurred. So now we’re at a weird point where we are evaluating the trade off between money, progress, and human lives.

Unless Tesla cars ignore the action of the drivers, those crashes are caused by the drivers who do not pay attention to the road. I'm no Tesla or Musk fan but blaming Tesla for drivers actions is not ok.

Reply on news.ycombinator.com