Why Tesla's autopilot can't see a stopped firetruck

This problem has been studied to death in the airline industry. Surprising to see this repeated all over again.

Clearly it's possible to use these features in a way that increases risky distraction, but I think it's a mistake to assume such a driving mode can't be safer on net when used properly.

This is the common case in pretty much all automated manufacturing plants where you have to hit the big red STOP button when stuff goes wrong. I don't quite see how its a failure when it has been working for them. In any case, do you have an alternative proposal that works?

Naturally there is always going to be human error, but it doesn't make sense to say "semi-autonomous" is bad while "full autonomy" is good. Guess what, we're going to have to deal with less advanced versions of the software before we can arrive at the real deal. The trick is to know and communicate the limits of the system. For example Tesla will alert you visually and through audio if you don't have your hands on the wheel. If it has to ask you three times, it'll just disable AutoPilot for the rest of the drive. It's just like a more advanced cruise control. If I hit a fire truck because I was on cruise control would anyone write an article blaming cruise control? No, because everyone knows the limits of that system. Crashes like this will force people to learn the limits of current autonomous software as well.

Better to completely remove the steering wheel, brakes and gas pedal than try for some half-way hybrid.

No human can stay alert enough to take split-second evasive maneuvers while doing nothing 3599 seconds out of 3600.

Perhaps Tesla, if the model has a camera, can detect where the drivers are - and if not consistently looking forward then alert them?

You know what's the solution? Don't use the feature if you can't use it. There are people that drink and drive, we didn't ban cars because of that...

If you can't achieve the same level of awareness or don't want to, well, too bad, don't use it. My SO stress when she drives, she is a good driver but she just doesn't like it. I'm pretty sure she will have enough awareness, but that thing will lower her stress considerably and will make driving much more pleasant for her.

What kind of ridiculous, half-assed logic is that? There could be any number of reasons they can't confirm this yet.

I haven't officially confirmed I'm not an alien being from a far away planet -- so that must mean I am one!

That changes the story a lot, do you know where you got that information?

https://www.youtube.com/watch?v=FadR7ETT_1k

I guess this was in ideal conditions or maybe the Tesla in the firetruck accident doesn't have this software update.

Basically, I rear ended somebody - but the details of why that happened are interesting to me.

I was going down a road I usually travelled to go to work that morning; two lanes each direction with a middle "suicide" lane.

Traffic was fairly heavy, as it always is in the area during morning traffic; there's a school not too far ahead in the direction I was travelling, and traffic "bunches up" and slows in the area, thanks to a crosswalk, people dropping kids off, and a "radar speed limit sign" that tells drivers how fast they are going in relation to the speed limit when school is in session (causes people to hit their brakes). Furthermore, not too far after the school is a stoplight for an intersection with another similar road.

Where I was at just before the accident was crossing a neighborhood street at a controlled intersection; also at that intersection, the suicide lane changes to an unprotected left turn lane.

So there I am, tooling along at my normal 40-45 mph (appropriate speed for the street I was on). Coming up on the intersection, a tractor-trailer semi was in front of me; the driver decided to make a left hand turn at the intersection, and got into the suicide-turning-into-a-left-turn lane. I decided to accelerate a bit to go around him. Now - I was kinda in an "autopilot" mode, not paying as much attention as I should have.

The traffic in front of me - thanks to the conditions I mentioned - which were typical of the morning time period, should have clued me in to "slow down" (which I have since taken to heart). That area, due to the traffic and conditions - is a "stop and go" situational nightmare. Traffic moves a bit, slows, maybe stops, starts again - and people do the typical: They don't keep their foot on the brake, so while they are "going slow" or even "stopped" (idling, not enough power to move) - their brake lights aren't "on".

So there I am - going around the truck - and "traffic ahead of me" registers, but I didn't see any brake lights, so while in my "autopilot" morning haste/haze/whathaveyou, I made the incorrect assumption that traffic ahead of me was moving; accelerating, maybe not going as fast as I was, but getting there. Which was the wrong assumption, of course.

The next thing I knew, I recall that the car in my lane ahead of me didn't have his brakes applied enough to turn his brake lights on, but wasn't moving anywhere near my speed; then he hit his brakes full (causing his lights to go on), and I slammed on my brakes just a bit too late in my realization. My speed dropped instantly, but not quick enough - I was probably going 15 mph or so when I rear ended him.

I was driving my 1996 C1500 pickup, his was a small crossover-like vehicle. Not much damage on my end; his bumper was mashed, lights broken, etc. We pulled over, exchanged insurance info, no cops involved, and went on our way. I called my insurance immediately, of course, and let them know what happened.

So it was a combo of things; my "autopilot" inattention, my speed, the traffic situation, the way people were "braking", going around a semi truck that was blocking my view then "road clear" ahead - it all came together in a "perfect storm", ending up with me rear-ending someone.

I tend to think that an autonomous car, or one with automatic braking (features not to be found on my truck at the time), would have been paying "more attention" and would have seen the car ahead of me not moving as fast (never mind the lack of brake lights), and would have hit the brakes and came to a stop much sooner. Indeed, had I been paying better attention, and going a bit slower (perhaps only going 30 or 35 mph), I could have applied my brakes much sooner, and avoided the accident.

Lesson learned, I guess.

It's why everyone else uses lidar instead, or in addition. Really, it has the same limits, but the wavelength of visible light is small enough that they're only an issue for microscopes.

We change out our phones every couple years, but may car is five years old and I'm just about to finish paying it off. With a decade being a reasonable replacement cycle for a car, how will the car technology keep pace? Surely people aren't going to start buying iCars as often as they replace iPhones. Even if you adopted a service-only module for the industry, it's not like driving services will want to replace their entire fleet every two years.

Will cars have some sort of "technology package" including the computer and some of the sensor modules that can be made plug-and-play removable so you can upgrade your car?

I'm not sure how many people _really_ think that; it's pure fantasy. To be honest, I wouldn't be surprised if they're not obsolete in 20 years. AI stuff has a tendency to get nearly good enough quite quickly, and then improve very little more for decades afterwards.

It's not a panacea of course.

All of which is just to say that those voices pointing out the technical challenges are up against seriously desperate interests. This could get messy.

My guess is that you're right: level 5 will take about as long as it takes to get to photorealistic rendering - never quite there.

Also, on the other hand not being able to recognize that your car is about to slam into a stopped emergency vehicle is also a giant gaping flaw on the driver's side.

Disclaimer to put my last statement into perspective: I have absolutely no relation to Tesla except for being a potential future customer. However I am driving a decent stretch daily and get to experience first person many stupid things people do in their cars due to ignorance/not paying attention/whatever, also I often use the adaptive cruise control of my Audi which shows me quite some false positives and restrictions (such as not letting itself deactivate below 15mph, heck, even deactivating itself without previous notification)

Also the problem is a road sign or overpass over a crest of a hill. It's stationary directly in front of you, the road simply curves away.

EX: Here is a car in-front of a road sign. http://iowahighwayends.net/ends/June06/16/34exit263_99_halfm... We assume the sign is not on the actual road, and the software makes the same assumption. It's just not always true.

EX2: https://i.pinimg.com/736x/f7/11/eb/f711ebb462449a422151daab6... again perfectly safe but it looks like a static object blocking the road up ahead.

I wouldn't hold my breath for that. Faster reflexes notwithstanding, automated systems don't change the laws of physics. If anything, I'd expect a human--at least during daylight hours--would be more likely to notice a deer at the side of the road and react accordingly before they ran in front of the car.

Move fast and break things indeed.

There are a ton of datasets out there for things like traffic sign identification, and vehicle/pedestrian/etc identification - but most of those vehicles in those datasets are:

1. From the rear (or oblique rear angles)

2. Of mostly automobiles and small trucks

There likely aren't many real-world datasets of vehicles facing the vehicle and stopped in the road, and/or of larger vehicles (buses, semi-trucks/lorries, emergency vehicles, bulldozers/tractors, etc).

Without having such datasets, the AI you have trained won't have any way of knowing about those objects, and they may be odd enough that they can't be easily extrapolated from the datasets you do have to train on.

And by "dataset" - I mean you need massive amounts of data; 50,000 images would be a "small" dataset. Ideally you want many times that, plus you want to "generate" synthetic views for more training data (by skewing/warping the existing images, it helps the system generalize on the data).

That's a possible explanation, but I don't know if that's the reason, part of the reason, or no reason at all. Plus, it's not to say they are "at fault" - or to absolve the driver of any responsibility.

I'm just throwing it out there, based on my knowledge and experience - which isn't much, I admit (I completed the Udacity Self-Driving Car Engineer Nanodegree last year, plus Udacity's CS373 course, and what is now the Coursera Machine Learning course - but I took it as ML Class in 2011 before Coursera existed).

I'll assume the latter: Even if you were able to perfectly maintain an accurate 3D model of the surrounding geometry (and associated trajectory of all moving objects), it doesn't mean you have enough information to make 100% accurate predictions. For example, a frozen over road often has very nearly exactly the same physical geometry and color, but extremely different behavior to cars trying to drive on it. Other weather effects like fog or snow bring their own challenges.

You're right though, that precise and accurate 3D models do do a lot to make driving easier and more correct, the problem is that the hardware necessary to produce those models is still very expensive (LIDAR). Most (all?) of the advanced self-driving prototype cars use LIDAR, while Tesla has decided that they can do without for their consumer models, sticking with just the simpler and less expensive RADAR and cameras.

In theory, Tesla's right, since humans get by just fine with only two "cameras", but we're clearly a long way from automating that level of performance.

These two things are treated differently by a radar system - imagine you are driving in semi AV mode and you come around a turn and there is a vehicle on the side of the road stopped (i've been in this situation). For a hot second the car is pointed right at that fixed object and is probably still processing the lane curvature (if it can at all, some adaptive cruise systems don't have lane monitoring). The vehicle will ignore the fixed object and keep going as to not slam on the brakes which can be very dangerous.

I witnessed an accident very similar to this. A vehicle was partially blocking the left lane of a 4 lane highway. A pickup truck was traveling at highway speeds in the left lane. A small 4-door car was following the truck too closely.

The truck saw the lane blockage fairly late and performed an emergency lane change to avoid hitting the stopped vehicle. The small car behind the truck slammed into the stopped vehicle without ever even hitting the brakes. The driver was very seriously injured(the car battery was in her lap and most of the engine had entered the cab) but afaik survived.

That was my life lesson to always allow space between me and the vehicle in front of me.

Ignoring things that aren't moving is a standard technique in radar to prevent your returns from being swamped by, e.g., the ground instead of fast moving planes you're looking for.