Hacker News Reader

Comment by simonsarris | Hacker News Reader

> What we know is that the vehicle was on a divided highway with Autopilot engaged when a tractor trailer drove across the highway perpendicular to the Model S. Neither Autopilot nor the driver noticed the white side of the tractor trailer against a brightly lit sky, so the brake was not applied.

Tragic no doubt, but I'm relieved that this was not a "Autopilot did something very very wrong" story.

Autopilot has the potential to save a large number of lives (I'm sure Tesla execs are thinking about touting "estimated lives saved by autopilot" if the numbers work out, after a few billion miles), so I hope incidents like this don't hamper public perception and therefore research.

View on HN

Autopilot engaged when a tractor trailer drove across the highway perpendicular to the Model S. Neither Autopilot nor the driver noticed the white side of the tractor trailer against a brightly lit sky, so the brake was not applied.

How is this not an "Autopilot did something very very wrong" story?

Tesla autopilot drove the vehicle full-speed into a tractor-trailer, which wasn't registered by its sensor systems. It's great that Tesla already has an explanation for the failure, but the situation as described is far from an improbable edge case. What they are saying is that the autopilot can't detect white objects when a bright light is present -- that's a pretty serious limitation to overlook.

Human drivers do this kind of thing with great frequency. As long as the system is at least as good as a human driver at this stuff, I'm happy with it. Actually, scratch that: as long as the system combined with me is at least as good as a human driver, I'm happy with it.

How do tractor trailers drive perpendicular on a highway in a probable case?

Still, I agree it is strange that a white/glossy object cannot be detected with radar. Is it a stealth truck?

Where I live some highways have a sign that says:

> CAUTION: SOLAR GLARE IN THE A.M.

It's serious enough to warrant its own road sign, but its not as much of a realistic issue on a divided highway, so maybe this road didn't have such a sign.

This type of danger is probably what the article is describing.

If the driver didn't notice it either, why is it Autopilot's fault?

The autopilot may reduce driver attentiveness, making the edge cases in its automation more deadly than they might've been without it.

I guess the question is, was the driver able to see it? I can see how maybe the contrast was so low that maybe it was impossible to see anything except for the wheels which you might not notice. And if the part of the tractor trailer in the path is elevated... sensors sweeping forward see just road.

I guess I'd like to see an NHTSB simulation of the situation in manual driving to see how humans do unassisted.

And if the part of the tractor trailer in the path is elevated... sensors sweeping forward see just road.

It could be even worse. If there was a "road surface mirage" then it's possible that both the truck and the road underneath it were not visible at all. Since the forward radar doesn't have the range or coverage to see a truck at a distance, the only sensor that could have detected the problem was the camera. Because of the phenomena I've mentioned above and the parent post has mentioned, we know this can't possibly work.

Even worse, the circumstances in which the car's sensor could fail are precisely the ones in which the Mark 1 Eyeball would also fail.

A google search shows several white trailers against blue skies. It's hard to see how a human could miss them unless there were weird conditions or they weren't looking that way. https://www.google.co.uk/search?q=semi-trailer&num=30&source...

Replace "autopilot" with "cruise control" -- would you have the same situation? Autopilot is basically advanced cruise control plus steering assist. From what I make of this story, this same accident would have happened with any other car with cruise control turned on.

A lot of modern cars have cruise control with radar for keeping distance to the car in front. Some use the same sensor to auto brake for obstacles at city speeds. I'm sure some will also alert for obstacles at highway speeds, but likely don't auto break then, for safety reasons.

I disagree. In a car with normal or even advanced cruise control, the driver would still be paying attention to the road and ostensibly would have seen the truck ahead and applied the brakes. Autopilot makes it very easy to not pay attention at all and let the car drive itself for minutes at a time.

I'm sure you'll say, "yes but that's a driver attention problem, not a technology problem", but to me they're the same thing. If you give drivers technology that encourages them to not pay attention then accidents caused as a result of that are partially your fault.

That's the same as blaming your car company for getting speeding tickets when you used cruise control. How many of those letters have you sent out?

No, it's not the same at all. With cruise control you are still fully in control of your car and necessarily aware of how fast you are going and what is going on in your surroundings. With Autopilot that is simply not the case - the technology virtually begs you to set it and then play with your phone or the touchscreen instead of paying attention to the road, text warnings be damned.

If you want an actually applicable analogy, I'll pull one I've used before. Say you buy an oven that has this great auto-baking feature that lets you just put some cookie dough inside it and then come back whenever you want, even hours later, to have fresh baked cookies waiting. That's how it's marketed - the AutoBake Oven. Except the manufacturer says that you're still supposed to watch the oven at all times in case something goes wrong. So you make some cookies with it and they turn out great, and nothing goes wrong. And hey, it's the AutoBake Oven, right? It says it will bake things automatically so you don't have to worry about them for hours at a time. So you logically decide to put some dough in there and then go the grocery store for a few minutes to get some items for dinner. And you come back and your house burned down because the AutoBake Oven lit itself on fire due to having subpar sensors and beta software in it. Now tell me you wouldn't file a lawsuit against the manufacturer.

Or maybe it's just a human behavior problem, and unfortunately it may take a few accidents of this type to train ourselves that AI assist doesn't mean we can lessen our attention.

Especially after hearing about incidents like this, you won't find me going through my email while letting "autopilot" do its thing. I don't think I would before this, but there's certainly a lower probability that I would now. I'm being trained.

The autopilot doesn't drive the car, the driver does. The autopilot is a help just like cruise control. Tesla are very clear (apart from the naming of the feature) that the driver must still drive the car.

The autopilot will avoid obstacles it can detect and the driver is expected to avoid others (since he has eyes on the wheel and eyes on the road the whole time).

I think they misleading thing about Teslas Autopilot I its current state is that they call it "Autopilot" while other manufacturers call this kind of assist features e.g "Lane Assist".

A distance-sensing cruise control can't be trusted not to crash into the back of the vehicle in front either, not least because it can't be sure it can "see" it. I suspect motorcycles and other small vehicles may fool radars just like this case seems to have been about contrast and colors in a camera sensor.

>The autopilot will avoid obstacles it can detect and the driver is expected to avoid others

I'm sorry, how is that supposed to work, exactly? How does the driver know? Sounds like the driver has to avoid every obstacle - to avoid the case where the autopilot fucks you, like this one - and in that case, what is the autopilot doing?

The driver has to be aware of every obstacle and observe the autopilot avoid them, or intervene should it fail. Yes.

Have you driven a car with a distance-keeping cruise control? It's very comfortable to let it do that and just watch it and take over when it fails (it's limited in the braking and acceleration it is allowed to do). I can't take my eyes of the road or my hands off the wheel for even a second but it's still useful because it removes all the micromanagement of throttle and brake and just lets me supervise.

That is a massive assumption to be making. That the driver can see and react to every obstacle. Not to mention there are videos where the Tesla reacted before and on behalf of the driver, saving them from an accident.

The "lane assist" (what it should be and is called by other manufacturers) spots any number of obstacles that the driver failed to see - takes over - and saves the driver from an accident. Or reacts faster than the driver could react - thus saving the driver. That is what it is doing. It is not driving your vehicle. It is assisting in driving your vehicle.

It is still possible to drown with a life vest on. That doesn't mean you shouldn't wear one because "if I'm expected to stay above water - what is the life vest doing?"

That's some PR firm-level explaining. First, autopilot is not just lane assist, it's lane assist + adaptive cruise control + additional features (it can change lanes if you tell it to). Other OEMs have purposely shied away from enabling an autopilot-like feature set together, even though it's technically possible, because they are absolutely terrified of something exactly like this situation happening.

Second, you're completely ignoring that autopilot will very much entirely drive the car by itself for minutes at a time (reports are anywhere from 5 to 15 depending on software version), which will obviously encourage drivers to not pay attention to the road. I don't care about disclaimers, I don't care about nag screens or chimes, and I definitely don't care about some warranty text that flashes on the screen. If you make a car that can drive itself you encourage drivers to let it do just that and you should be prepared for that eventuality. Period.

This autopilot is very new and fairly experimental technology. You're basically demanding that this not be true somehow.

Very new and fairly experimental technology that can kill you tends to be fairly heavily regulated. New drugs, rockets, aircraft, etc. don't generally get released via software update to the general public. They go through very regimented, restricted, contracted trials.

Tesla risks getting regulators to crack down on new car features across the entire industry if they're not careful.

Much like if you make a car that does 0-60mph in 3 seconds or does 200mph you encourage doing that. The fact that the autopilot is so good that it "almost" works as a driver is an issue for Tesla as we can see here. I realize autopilot has a nice ring to it, but it's not a chauffeur...

Tesla sure has balls to pick the name "Autopilot" when it really should be called "CoPilot" and that would have saved a lot of trouble.

> The fact that the autopilot is so good that it "almost" works as a driver is an issue for Tesla as we can see here.

If only someone could have foreseen these issues...

http://www.theverge.com/2016/4/27/11518826/volvo-tesla-autop... http://www.thedrive.com/news/3753/former-google-exec-and-ai-... http://www.thedrive.com/tech/3909/volvo-rd-chief-teslas-auto...

The Drive article is an eerily prescient interview with Volvo's R&D chief and is only two weeks old:

     Critics see a neither-nor limbo, with technology just
     good enough to make drivers drop their guard, but not
     good enough that driving off a cliff is an
     impossibility. ... Shooting for but missing Level 3
     autonomy would create his nightmare scenario: a car
     that gets flummoxed by one of thousands of variables,
     then asks a daydreaming co-pilot to wake up and save
     the day.

> The autopilot doesn't drive the car, the driver does. The autopilot is a help just like cruise control. Tesla are very clear (apart from the naming of the feature) that the driver must still drive the car.

I'm sorry, but that's a cop-out. What's the point of an autopilot if you have to avoid every single obstacle and react to every single thing as if the auto-pilot didn't exist? You can't say "autopilot is safer than normal humans" (heavily implied by the blog post) and then turn around and say "but it's really the driver who's responsible for everything!" anytime there's an accident.

Unlike autopilot, cruise control has very limited functionality. No driver expects it to do anything other than keep their speed, so they are still actively engaged with the road to respond to anything that takes place.

> I'm sorry, but that's a cop-out. What's the point of an autopilot if you have to avoid every single obstacle and react to every single thing as if the auto-pilot didn't exist?

You don't have to react - you have to supervise it though. It lets the driver monitor a system that micromanages steering, brakes and throttle, instead of doing that micromanagement himself. It requires the same attention but is less work.

> Unlike autopilot, cruise control has very limited functionality. No driver expects it to do anything other than keep their speed,

Well, these days you can often set it with a distance to the car in front using a radar. You then let the cruise control do something more than just keep constant speed, it will slow down if the car in front does etc. The driver has to pay full attention though for anything the cruise control can't handle.

You seem to imply "what's the point of an autopilot if I can't take a nap". - well the issue is that it's a little too good to be called a cruise control, but not good enough for a nap. We're simply not there quite yet. At least not legally.

The problem to me seems to be: at highway speeds, you have to decide very quickly whether or not it's something that the system can't handle.

If you're constantly responding to every change in conditions, what's the point of the autopilot? So you don't respond every time, and then something comes up that the autopilot doesn't recognize, and you don't know that until it's too late.

> If you're constantly responding to every change in conditions, what's the point of the autopilot?

The same as a cruise control - you are supervising rather than micromanaging because micromanagement is more work than supervision. A distance-keeping cruisecontrol may be changing the throttle level dozens of times per minute. A driver supervising that system can keep an eye out for stalled cars on the side of the road, wildlife etc., withput using most of the attention on keeping a constant distance to the vehicle ahead. This complements the driver and increases safety because the drivers cognition can be used to what it's best at, and simple tasks like keeping a constant speed or constant distance can be done by a computer.

The problem with this "Level 2.5" autopilot is twofold:

1) It's called "AutoPilot", even though that shouldn't be the name of it until it's feature complete (level 4 autonomy) simply because people seem to assume that AutoPilot means something that drives by itself.

2) It appears good enough to allow the driver to not pay attention, while still requiring the driver to pay attention. The disclaimers that the driver can never be distracted for a short momet (i.e. use it like a pilot uses an autopilot) don't seem to help.

A lot of the comments here seem to indicate "if I can't text/sleep/whatever then what's the point of an AutoPilot" - clearly misunderstanding the current capability of the system!

I think we're in agreement. My point is that you can't supervise something you don't understand so I don't know how a driver can reasonably be expected to switch from supervision to reaction quickly enough every time that is needed.

I know what my cruise control will do (effectively nothing). I don't know how to anticipate what Tesla's sensors and "AI" can't handle.

Calling it 'autopilot' is a very bad thing, then, because 'autopilot' means 'runs automatically' to the general public.

And even though actual autopilots in planes are monitored by meatpilots, meatpilots can still take their attention away for a short time. The autopilot keeps the plane flying straight and level, and doesn't take on any navigational role (in most cases). Unless your roads are dead straight and there's no traffic, driving requires a constant navigational role.

> Calling it 'autopilot' is a very bad thing, then, because 'autopilot' means 'runs automatically' to the general public.

Indeed. Tesla wanted to use that name now even though it's the proper name for a feature not yet complete. They have to use a ton of warnings and disclaimers to do it. We'll see if authorities find that sufficient. If I were tesla I'd rename it to something with "assist" for the time being.

> And even though actual autopilots in planes are monitored by meatpilots, meatpilots can still take their attention away for a short time.

So does Teslas autopilot, but the time you can safeley take your attention away on a highway may be perhaps a second or two if you are on a clear straight with wildlife barriers. The autopilot will ensure the car runs straight while you fiddle with the radio.

The same time in an airplane may be ten minutes. THAT is the difference. In an airplane theare are controllers making sure there is no traffic one in front, and there is no risk of a deer or a child stepping in front of the plane. Basically the AutoPilot will have the meatpilot respond to anything it can't handle too - which in an airplane is mostly tecnical issues.

> Tesla autopilot drove the vehicle full-speed into a tractor-trailer

The tractor trailer drove across the divided highway without seeing an oncoming car. I've seen this accident happen in front of me with a fatality, no auto-pilot involved.

A vehicle moves in front of you when you're sufficiently close and you've no time to react. Anything (like glare) that makes such a vehicle harder to spot reduces your available reaction time.

Making it sound like there was a static obstacle in front of the car, and it drove into it at high speed is very tendentious, and not warranted by the report, I feel. It is exactly the kind of 'omg AI kills driver' fearbait that we don't need.

If self-driving systems were ten-times safer than people, then there'd still be a lot of people being killed in them. We need to get a sane perspective on their risk.

It is essentially AI kills driver. The truck was making a left turn in front of the Tesla, which was in the right lane of a 4 lane divided highway. The truck would have had to accelerate in the left turn lane, cross the divider, cross the left lane, then drive further so that the middle of the trailer was in the right lane. This takes at least 5-6 seconds for a semi, and if the semi were detected, there would have been ample time to apply the brakes to give the truck enough time to clear the intersection. If the autopilot system can't detect and respond to this situation appropriately, it shouldn't be able to be activated on a non-controlled access highway.

Yeah i can't believe this is not bigger news. The Tesla AI was almost good enough to replace an attentive driver. But almost is not good enough.

> replace an attentive driver

It's unrealistic to assume that human drivers are attentive all the time. Autopilot never gets tired, never eats, never applies makeup, never sneezes, and never gets distracted by screaming children.

You have to compare autopilot to realistic humans, not your platonic ideal of what a driver should be. I'm sick and tired of people making statements about road transit predicated on the idea that real humans pay attention to the road with 100% fidelity at all times.

I read it more closely and they do try to enforce that a driver has his hands on the wheel, etc. I am not convinced of the benefits though. I think something that enforces people to pay attention when driving is a better solution.

> It's unrealistic to assume that human drivers are attentive all the time...

Yes, this is an unfortunate fact of life that human beings are no angels.

However, most modern societies agree that reckless drivers are to be held accountable for their less than perfect behavior. If a police officer (or more recently, surveillance camera) catches you while DUI, or texting, or some other of the most blatant irresponsible behaviors, they will give you a big ticket. And if the consequences of your own idiocity happen to materialize you will suffer much sterner punishment (up to criminal charges for manslaughter, if it came down to that).

That's assuming the infractor does survive. It is a known fact that a non trivial amount of people responsible for accidents end up loosing their lives (or those of loved ones) in the very tragedies they caused themselves. Death is the ultimate judge, and a merciless one at that.

> You have to compare autopilot to realistic humans, not your platonic ideal of what a driver should be.

Why?

Neither the engineers who created those subpar systems - nor the executives who pushed for an unfeasible schedule, nor the shareholders that benefited from the whole deal, - will ever face the same consequences as described above. Corporations raison-d'etre is precisely to shield people engaging in economic activities from the unintended consequences of their actions; that's why they must be held to a higher standard than the everyman down the road.

Would you imagine Tesla's Board of Directors having approved the green light for this impressive-yet-still-limited technology, if the company was required to pay for every traffic ticket that every Model S accrued while being in autopilot mode, nationwide? What about being accountable for every accident, no disclaimer licensing or such nonsense?

> I'm sick and tired of people making statements about road transit predicated on the idea that real humans pay attention to the road with 100% fidelity at all times.

People will make statements about road transit, because it is literally a life-or-death situation. Maybe you do prefer the cattle to shut up and die quietly, so your utopic techno-fantasies are never challenged?

I prefer to talk about real utilitarian tradeoffs instead of putting I imaginary, arbitrarily virtuous angels against computers. Every day you delay automation of automobile traffic is another batch of lives lost.

Are you more interested in saving lives or in expressing the short sighted and sanctimonious outrage of a Luddite?

Since when is caring about Engineering standards became Luddite?

How many lives do you think autopilot is going to save if being as good as a sleep deprived teenager is good enough? And who benefits from that state of affairs?

Anyways, you are a true beliver or a paid shill, so have a good night. EOM.

> Since when is caring about Engineering standards became Luddite?

Since you impose unrealistic standards on progress in an attempt to bolster your own moral position

It's also unrealistic to assume that an attentive driver would have avoided the crash.

Driving at speed, you have to make assumptions that other people will do the right thing. You pass cars all the time. If you assume that the car might side-swipe you, you wouldn't get far, and you'd be a hazard yourself.

You drive on undivided highways all the time without driving as if the oncoming car might cross in front of you. And so on. We have to assume that others are minimally competent. That's the point of road laws, not just to control you, but to let you assume things about others.

I think that your conclusion is disingenuous.

Where I'm from if you make a left turn and get side-swiped then it's your fault, period. You made an unsafe turn, plain and simple[1].

If what you describe is true, then human error is the main cause of the accident (although the AI failed to prevent it and absolutely should share the blame). I just think "AI kills driver" is a bit sensationalist don't you?

[1]Of course assuming that all other vehicles present were obeying the law (e.g. not speeding) and there were no other extenuating circumstances.

Autopilot doesn't need to be perfect, it just needs to be better than the average human driver.

On average, humans are rather unsafe car drivers.

The only situation I can imagine a driver not noticing a huge truck perpendicular to them on the highway, regardless of what color it is, is that they're not paying any attention at all.

This peculiar wording is making it sound like the computer made the same mistake as the driver, but it seems much more likely that they made two different mistakes. The computer's mistake was in image recognition, and the person's mistake was trusting the autopilot fully and not paying attention to the road.

The only situation I can imagine a driver not noticing a huge truck perpendicular to them on the highway, regardless of what color it is, is that they're not paying any attention at all.

The optical distortions that cause road surface mirages could well entirely hide another vehicle. In this case, since the trailer was painted white, only the wheels and lower chassis would need to be hidden for the image to become unrecognizable. Not saying that's what happened in this case, but there are situations when your forward vision could be compromised. (There is a theory that the colder ocean may have caused a similar optical phenomenon and hidden the iceberg that sunk the Titanic.)

That doesn't sound like what happened. "The driver didn't notice" well, that's why they're using autopilot.

No matter how many messages people throw at users, any autopilot that doesn't detect that the driver is awake and operating the vehicle is going to be used by sleeping or otherwise unattentive drivers. It's just human nature.

So what likely happened was the driver was screwing around and the autopilot completely failed to notice a giant white truck crossing the road and blocking their way until it was too late.

edit: reply is quite right, shouldn't speak ill of the dead that way. "Screwing around" is unfair. I just mean that "driver falls asleep/paying attention to other things" is the expected outcome of any autopilot system, and so it's unreasonable of autopilots to have any expectation of drivers for attentiveness.

From the information given, we don't know if the truck pulled out right in front of the car giving the driver no time to react, or if the truck pulled out a minute beforehand and the driver was just oblivious. In the absence of any information either way, it's nasty to just assume the driver was inattentive.

I don't think it's nasty, I think it's realistic. I also think that it aligns with the research that shows humans take a significant amount of time to adjust to taking back over control from autopilot. This is one of the primary design considerations behind the Google self driving car having absolutely no steering wheel or brake. The illusion of humans being able to be the fallback when they're not physically capable of it is a dangerous situation.

I don't think it's realistic to rampantly speculate about a situation where so little information is available. And I absolutely think it is nasty to talk ill of the dead when there is zero evidence for what's being said.

For all we know the driver had half a second to react before he died. Or maybe he had five minutes. The point is, we don't know. To start from that position and say it's "likely... the driver was screwing around" is pretty crappy.

Fair, I wasn't reading it as a knock on the driver. I read it as a mistake for Tesla to hide behind the driver having ultimate responsibility when the reality is no driver is going to actually be attentive or even capable of taking over.

I'll admit that the word "screwing around" was unfair and judgemental, but I'm gonna be honest: I would never want to own a car with this sort of "autopilot with driver fallback" feature because I'd totally fall asleep at the wheel. Completely. So I didn't mean any judgment by it. I could have made the same mistake.

In fact, I'm judging the software, if anything. Any autopilot software that doesn't force the driver to be attentive is effectively taking the responsibility for driving onto themselves, and so cannot blame the driver for being inattentive. You can't say "I got this" and then at the last second say "oh shit, I need you driver!" even if you put warnings that you don't really got this, because you're dealing with human beings who do normal human things.

Yes, I shouldn't have said the driver was screwing around, you're right. But if the driver fell asleep, and a conscious driver would have avoided this accident? I still think it's Tesla's fault, because he'd be awake and alive in a manual car.

And how many times do human drivers fall asleep in cars with no automatic driving features of any kind, and get themselves killed in a situation where Tesla's Autopilot would have saved them?

I'd recommend trying it before you knock it. I had similar reservations, but after using it for nearly a year, I find that I'm more attentive and more awake with it than without it. A couple of weeks ago the system refused to engage (broken sensor somewhere) and I had to drive 80 miles home manually. It was exhausting! The mental effort of staying between the lines and maintaining a safe distance from the next car is pretty significant. Getting rid of that frees up more of your attention for the bigger picture. Now, I'm sure for some people it just frees up more of your attention for texting or sleeping, I'm sure I'm not representative of everybody, but I think it can easily be a net win for many.

I always pay attention while Autopilot is engaged, but I'm able to look farther down the road and take more care when checking out traffic to the rear and sides than I can when I'm doing it all myself.

>>This is one of the primary design considerations behind the Google self driving car having absolutely no steering wheel or brake.

Didn't know about this. If they are truly testing their software in a car without break and steering wheel I have a whole new level of respect for them.

When you start to design a self-driving car the question of a human fallback shouldn't even exist. The car either drives itself, or humans do. Mixing responsibilities in situations of life and death doesn't end well.

A car traveling 60mph is traveling 88 feet per second - double this because the truck is likely traveling the opposite direction at the same speed. Average human reaction time in best case scenarios is ~215ms (see http://www.humanbenchmark.com/tests/reactiontime).

A Tesla Model S P85D can brake from 60-0 in 102 ft. (see http://www.edmunds.com/tesla/model-s/2015/road-test-specs/)

Assuming the driver is perfectly attentive, and hits the brake immediately upon noticing the truck, they would still need at least 139.84 feet to come to a stop.

This also assumes the truck is no longer moving. If it is you'd have even less distance to stop.

I don't care whether you're waking up from sleep, or completely alert and instantly react, it's a huge assumption to think that any possible reaction would have prevented an accident or fatality. It's just as likely that even had autopilot engaged the brake immediately, the truck still would have plowed into the Tesla and caused a fatality.

tl;dr - two way traffic involved in a head on collision is a dangerous game, made doubly dangerous because of the combined speed of both vehicles working against inertia.

From the article, it seems that this autopilot checks that the driver is attentive.

> When drivers activate Autopilot, the acknowledgment box explains, among other things, that Autopilot “is an assist feature that requires you to keep your hands on the steering wheel at all times," and that "you need to maintain control and responsibility for your vehicle” while using it. Additionally, every time that Autopilot is engaged, the car reminds the driver to “Always keep your hands on the wheel. Be prepared to take over at any time.” The system also makes frequent checks to ensure that the driver's hands remain on the wheel and provides visual and audible alerts if hands-on is not detected. It then gradually slows down the car until hands-on is detected again.

It does but it's not terribly frequent and it's easy to bypass. When you give people a feature they're going to use it however they want regardless of what you tell them. Users are funny like that :)

What if you get into a collision while you are reading all the text in that acknowledgement box?

I don't think the setting can be changed unless the car is in Park.

Those "frequent checks" are pretty rare. I've gone 15-20 minutes without seeing one. For a while it was every 3-5 minutes, but they've dialed it back up in newer software updates.

>detect that the driver is awake and operating the vehicle

RTFA. It mentions explicitly that autopilot checks for hands on the wheel.

This is a very weak response.

And yet it really doesn't. Every so often it'll beep at you to make your presence known, but it can easily go many minutes with hands off the wheel and no checks.

I'd say most accidents are the result of the driver "not noticing", especially in the current majority of cars which lack autopilot, so even if this edge case exists in the future, having an autopilot is still an improvement.

  > That doesn't sound like what happened.

Everything is speculation at this point so I am not sure how you can assume that's not what happened.

  > "The driver didn't notice" well, that's why they're using autopilot.

If Tesla builds and designs a system to be used as an assistant, explicitly tells and warns the user of it's use case and potential danger of using it as anything other than an assistant, then the excuse that "it's human nature" is null, I don't think it's human nature to leave your safety and well being at the hands of something designed to be nothing more than an assistant.

Perhaps the name autopilot exerts to much confidence in its capabilities.

I'll give you that it's not a story of Autopilot driving off a cliff, but it may be a case of someone getting killed due to a problem with Autopilot. While it's factually true that the driver did not notice, what we can't know is if the driver would have noticed if they were more aware without autopilot engaged. Anyone that thinks drivers pay as much attention to the road when Autopilot is enabled is being naive.

Technologies like autopilot are going to save lives. This was one of around 90 fatal traffic accidents in the US on the same day (based on historical averages). It's going to be hard to make a foolproof system, however anything that makes it less likely for an inattentive drive to crash into something is a win.

How would we ever know that the autopilot acted correctly? Should companies be required to release logs after every crash when a car is in autopilot? Seems fair.

Black box? But logs must be useful for the insurance claim (by either the manufacturer or the driver).

If autopilot fails when it's sunny and the obstacle is white, I have doubts about autopilot.

Humans fail for even more stupid reasons.

This seems like a pretty big oversight, large object that car is too tall to clear safely seems like it could cause a lot of problems. Would the car try to make a lane change while next to a trailer?

Tesla's Autopilot only changes lanes when commanded to by the driver. This is done by putting the turn signal on. If you put the turn signal on while next to a trailer that doesn't have stuff coming down near the ground, then yes, the car will likely change lanes right into it. But it's easy to avoid: just look before you signal.

Unless it's changed recently, drivers have to tell the car specifically to change lanes (using the turn signal) or else it won't do so. And yes, this is the same issue as that other fairly recent incident in which a Model S drove itself into a parked trailer when the owner summoned it (supposedly inadvertently) due to the height of the trailer bed above the ground. Very concerning to me given the prevalence of trucks on the road.

Reply on news.ycombinator.com