Preliminary NTSB report on Tesla crash
ntsb.gov
ntsb.gov
While the NTSB don't typically investigate car accidents, they do for transportation (truck) accidents. It's likely that the NTSB investigated because it involved a truck with a possible systemic issue, with cars going under trucks, and the Tesla autopilot being a factor.
One noted NTSB rule is that Tesla or other parties [0] do not comment regarding the NTSB investigation, except with the NTSB senior investigators' permission.
[0] http://www.ntsb.gov/legal/Documents/NTSB_Investigation_Party...
http://www.bloomberg.com/news/articles/2016-07-08/driver-aut...
Also it's not actually a gag order. I can't find any actual legal regulations which leads me to the assumption that this is less an NSL style "comply or you will be silenced" sort of situation. All the evidence I could find makes it seem like more of a "gentleman's agreement" developed over the decades of interacting with an extremely small pool of actors. ( American Railroads + American Truck Makers, and Airlines operating in US airspace ) Infact I grabbed this exemplary ( if a little long ) quote off the NTSB website from a press release regarding the FAA accidentally releasing investigation information when complying with a FOIA request before the investigation was complete. ( The NTSB don't care about the FOIA request, just that you wait until their job is done before answering it.)
>> The NTSB depends upon full participation and technical assistance by the parties in our accident investigations in order to ensure that our investigations are objective, rigorous, and complete. Allowing any party to release investigative information without approval may enable that party to influence the public perception of the investigation and undercut the fairness of the process.
>> Accordingly, we require that any release of information related to an ongoing accident investigation be coordinated and approved by the NTSB prior to its release. When the investigation is complete, these restrictions no longer apply.
The side of the trailer is corrugated metal painted white. It's not a smooth white surface. The vision system should have been able to range that.
Part of my work deals with medical patient safety but I study all sorts of safety including airlines, nuclear power, oil & gas drilling and refining, ....
There should be a mechanical fail-safe if the power cuts in this case.
Additionally automatic transmissions have a transmission lock, but that won't work while the vehicle is in motion.
Some modern cars use electric systems for both, I'm not sure how that would work.
And the power braking system, being pneumatic IIRC, keeps working for a couple of hard stomps on the pedal even if the engine stops running and you lose 12V.
Pretty much the only thing you can expect to lose is the ABS. Even then, I understand that system has a failsafe such that it keeps the car from spinning in the event of malfunction and brake lockup. You can see this in ABS-related accidents as straight skidmarks. But I don't think that works when you've lost electric power.
Edit: actually, the recent Koenigsegg One:1 high speed crash (driver not hurt) during testing at the Nurburgring was an ABS sensor failure, you can see the hallmarks in photos. Koenigsegg also deserve big props for having been completely open about it.
And drift cars use a separate hydraulic brake attached to the rear disks.
Others (usually less expensive cars) have a set of drum brakes inside the disk brake that act as emergency brakes.
If you have rear drum brakes, it's the same as my first example. The emergency brake activates the normal braking system.
If your battery dies while the car is running (say, even, that something causes a physical disconnect between the batteries and the rest of the car — a wiring fault, or whatever). Ideally you would be able to pull to the side of the road while your vehicle coasts, braking as necessary.
If the system detects a power disconnect and instantly engages all brakes, does that help or harm? Additionally, since powered items like anti-lock brakes are now unavailable, how hard should the brakes be engaged? Fully? Slightly?
Slamming on the brakes in a failure scenario is not automatically the right answer. Odds are it's probably the wrong answer more often than not.
Are they unavailable? Electric cars have a main traction battery plus the conventional 12V battery that all cars have.
A motionless 2-ton rock in the middle of a busy interstate because of a power blip is a terrible idea. And again, how hard exactly should the system brake? Pick a value between 0% braking and 100% braking that brakes maximally without locking up the brakes.
A 2-ton brick spinning down a busy interstate because the brakes locked up is arguably even worse than a motionless one.
In addition, the throttle control went through a pull cable device with an electromagnet. With the electromagnet on, a servomotor could operate the throttle. The emergency stop system would drop power on the electromagnet if the stall timer timed out, or on some other fault conditions. That forced the throttle to idle.
Then we had an Eaton VORAD radar. That data went into the main mapping system, along with LIDAR data, but it also was processed by a simple separate process that computed time to collision from range and range rate, and if it didn't compute a safe distance, or didn't reset the watchdogs, tripped the emergency stop system. If this happened, the LED sign on the back of our vehicle displayed "COLLISION IMMINENT".
This happened once during the Grand Challenge preliminaries. Several vehicles were in the starting gates side by side. We were ready to go, all systems running and armed, waiting for DARPA to release the hold signal they were sending by radio. The organizers decided to release the CMU vehicle first, and it came out of the starting gate and cut in front of our vehicle. The safety systems tripped and "COLLISION IMMINENT" appeared in the sign. After a few seconds, with the threat gone, the system reset and the sign went dark.
This was all fully automatic. There was also a remote engine kill system, required by DARPA.
We didn't win. But we didn't crash or hit anything. There were Grand Challenge entries that ran away, including, in 2004, one from CMU. Another one ran away because they filled their disk with logging info and this stalled the software. Steering and throttle froze, and the vehicle ran away until it hit something.
If you work on automatic driving, you have to prepare for trouble like this.
I'm not sure if the vision system is smart enough to be used for collision detection. The radar and ultrasonics are used to detect vehicles, but I believe the camera is just used to help follow the lines painted on the road.
Could it be that this trailer was out of spec? Could that be why 1. there is little damage to the truck, 2. The body is mostly intact but the roof is gone, and 3. the car did not detect the obstacle?
[0] http://www.sparebumper.com/index.php?act=viewProd&productId=...
(look at the NHTSA crash test videos on youtube)
It might be that cameras introduce enough noise and artifacts into the mix that its not worth integrating them at this point.
I don't understand why the truck wasn't detected. Not exactly a small target.
In a different domain, nuclear power plant design, the Three Mile Island plant had a solenoid activated valve for letting high pressure out of the reactor vessel. The problem was that although the plant operators said set the valve to close, the valve was stuck open. It turns out that the control panel light signifying that the valve was closed simply displayed the signal sent to the solenoid, not that the valve had actually closed. No secondary valve, no meter to determine flow.
There had been a operator's manual change for TMI and other plants of the same design, but the proper hardware change was never made. Hence TMI.
But I worry about the kind of thinking as in TMI and also in Tesla.
In the case of Tesla, they have to engineer so that the cars knows about obstacles before crashing into them. The fact that Tesla did not think through this pretty obvious scenario is frightening and suggests a fault in their overall process in engineering. I just feel that Tesla cannot be trusted in terms of engineering unless they have some very smart group check their design.
I doubt BMW would have made such a mistake.
For all of his self-promotion and as head of a car company and a rocket company, Elon Musk has never worked as an engineer for a large firm where he'd be mentored in engineering.
For example, the CEO of GM is an engineer (EE) who was mentored in engineering. https://en.wikipedia.org/wiki/Mary_Barra
And the same for IBM: (EE & CS) https://en.wikipedia.org/wiki/Ginni_Rometty
That looks a tad like fear mongering.
Please be specific. What is the fear mongering? It is a statement from someone educated as an engineer and trained as an engineer.
Engineering, whether it is designing buildings, cars, airplanes, is thinking through the scenarios and testing for them. This seems to be a situation where they were ruling out overhead signs, but because of trucks clearance of these signs would be well over 10 feet and they could check for anything under 10 feet or 8 feet and should have.
And, as I said before, if they didn't think this through, what else that we can't see are they not thinking through?
It's not like they haven't thought this through. Tesla has been clear that this is a Driver Assistance, not a full autonomous car. There are multiple warnings, including every time you enable auto pilot that makes it clear you must pay attention.
This accident while tragic from current reports looks like the driver was not paying attention to the road. The best way that we're going to get to fully autonomous cars is to collecting real world data and I think auto pilot has been a reasonable approach in that direction.
If by that you mean you're a software engineer, I do think it's fair to distinguish between getting an EE/CS degree vs say a BA in CS plus programming experience.
(I write this as someone in the latter group.)
The meta point I'm trying to share is that if you're trying to convince a technical audience(which HN certainly is) berating people by saying "I know better because I have this slip of paper" is the quickest way to get someone to dig in and dismiss your idea.
Engineers are natural skeptics, arguing from the technical side will always be the stronger position.
Basically you're saying that some level of collateral damage is acceptable, and because there's a warning, it's the driver's fault.
To be clear: if someone not paying attention causes a crash, it is their fault. This does not, however, absolve Tesla (or any other manufacturer) from the responsibility for releasing an insufficiently-tested and potentially dangerous system into an environment where anyone who has a passing familiarity with human nature knows it will not be used responsibly.
Tesla calls it 'beta' software, and went so far as to describe the deceased driver as a tester in its press release after the crash. Again, anyone who understands human nature can see that this is a cynical attempt to manipulate the public's opinion. It may, however, come back to bite them, when people start asking WTF they were thinking when they put beta software on the road in the hands of ordinary drivers.
Multiple warnings would seem to me to be insufficient to reduce the hazard presented by the Autopilot functionality; indeed, there are any number of videos of Tesla drivers using the vehicle contra to the warnings. As one specific example, consider that the Tesla Autopilot cautions the driver to maintain hands on the wheel[0], yet does not enforce this requirement[1] despite having the capability to do so[2]. There's also a problem with Musk viz. marketing: he's the very public face of the company, and he is frequently overly optimistic in describing the car's capabilities by blurring the line between current and future capabilities, e.g., implying that holding the wheel isn't critical with a wink-wink, nudge-nudge[3].
Tesla's PLM certainly has some sort of mechanism to continuously examine the risk analysis for the car, yet the Autopilot functionality doesn't seem to have been significantly updated to incorporate the changing risk profile. To add on all of this, why does Autopilot allow one to speed? Why can one enable Autopilot on a road such as the exemplar if it is contrary to the instructions and the car is capable of knowing the difference? What does the risk analysis say on the topic of the feature name "Autopilot" being misunderstood by the public? &c. &c.
I do wonder about the engineering processes at Tesla. I admittedly don't work in automotive, but I do work in a regulated industry, and Tesla's apparent engineering process makes me very uneasy. Risk analyses that I have done took into account that the user may not have read the instructions for use, and I struggle to understand how Tesla could not do the same.
[0] "Drivers must keep their hands on the steering wheel." https://www.tesla.com/presskit/autopilot
[1] "We drove for 10 miles without the message appearing." http://www.teslarati.com/what-happens-ignore-tesla-autopilot...
[2] It's not clear that they have a capacitance sensor as on properly-equipped Mercedes, but Teslas allegedly can detect minute torques applied to the steering wheel as happens when the wheel is held. I recall reading this about the Teslas, but can't find a citation at the moment.
[3] “It works almost to the point where you can take your hands off,” Musk laughs, “but we won’t say that. Almost.” http://www.wired.com/2015/10/tesla-self-driving-over-air-upd... also, "But by April, he told a conference that Autopilot was "almost twice as good as a person," even in its first version." and from the same, "Musk himself has retweeted news reports showing drivers using Autopilot with no hands on the wheel." http://www.autonews.com/article/20160705/OEM06/160709956/tes...
1. Can the fact that the driver is not required to maintain physical control of steering implements cause the computer algorithms to be considered legally culpable (and, by extension, Tesla) for failing to act as a driver is legally required to under Florida law?
2. Does the statements of Elon Musk et al imply that the car is fit to act as a driver in its legal responsibilities under the rules of the road (independent of any legal ability for a driver to discharge those responsibilities to the car), so that failing to, say, be able to detect an obstruction in traffic constitutes a defect such that it's violating the implicit warrant of merchantability?
That looks a tad like denial, especially in the light of your follow-up.
There are sound arguments to be made here. There's no need to make it personal.
If anything uncontrolled intersections and high speed difference merges (200ft radius 270* on ramp I'm looking at you) more efficient to handle with driverless cars, provided they're capable of identifying the road (easy) all the non-static participants (hard).
I'm thinking that a very good visual AI would be necessary to make distinctions like this; radar won't even see a wire, I don't think, and to lidar, it would look a lot like the grass. A touch-sensitive coating on the car would probably be a good idea too, so that the car can tell when if it is starting to scratch its paint on what it thought was an insignificant obstacle, like a branch.
Simply not looking at obstacles in one area because of the difficulty of rejecting false positives is a terrible idea, and demonstrates that that AI is not ready to drive.
"Radar tunes out what looks like an overhead road sign to avoid false braking events"
When I was developing Grand Challenge software, I used to have an Eaton VORAD looking out my window at home, tracking vehicles going through an intersection. It could see cars, but usually not bicycles. Range and range rate were good; azimuth info was flaky. Stationary objects didn't register because it was a Doppler radar. Output from the device was a list of targets and positions, encapsulated in Serial Line Interface Protocol.
The big problem with these radars is not seeing the road itself as an obstacle. When you're moving, everything has a Doppler radar return. Usually, the road is hit at such an oblique angle that it doesn't reflect much. But there are exceptions. The worst case is a grating-floor bridge.
LIDAR isn't a panacea. The charcoal-black upholstery used on many office chairs is so non-reflective in IR that a SICK LMS can't see it at point-blank range.
[1] http://www.fujitsu-ten.com/business/technicaljournal/pdf/38-...
The intersection appears to be at the bottom of a hill, not the top, as I expected based on Tesla's account re: white truck vs white sky.
I'm no traffic accident reconstruction expert but it appears to me that the top of the truck would have been below the horizon. Indeed, the photo is taken looking east, the direction the Tesla was traveling in.
https://www.google.com/maps/@29.4119817,-82.5444142,3a,60y,1...
If you squint or zoom-in, the light square on the right hand side of the road is in fact a vending-machine-stocking truck parked at a gas station on the other side of the intersection where this all happened, serving as a useful feature-height comparison; the semi was probably a little larger than that but similarly its wheels were likely underneath the horizon provided by the hill of the road. You can therefore imagine that more than half of the trailer might have been under the horizon of the hill, including wheels etc. -- frustrating any sort of machine visual inspection, 'cause that's even difficult by human eyes.
This Google Map image is 440m away from the actual crash site, which at 74mph corresponds to about 13.3s of reaction time. (The extra time from going at 65mph would would have only been an extra 1.8s.) It was intentionally chosen to be like "here is where a driver travelling East would have had no idea that there was a semi beginning to turn, pulling out onto the road in front of him."
The other data point needed is "here is where an attentive driver would have had no excuse for not knowing there was a semi in front of him;" that's about here:
https://www.google.com/maps/@29.4115366,-82.5429196,3a,60y,1...
This is 360m away, giving at least 11s of advance notice to the oncoming driver. Even with 5s of reaction time before he slams on the brakes, that should have been plenty of time. Clearly the inattentive driver is therefore a huge problem here, and the driver was presumably inattentive because he had been goaded by the ease of the driving system.
It's much harder to tell whether the semi should have seen the Model S and yielded right of way with this setup; as mentioned at 440m or so the Tesla should have been likewise invisible to a truck-trailer, so that's presumably when he would have started turning; it's not clear to me whether he would have then seen the car before his truck-trailer entered the oncoming traffic and he would have had no better option than stepping on the gas.
But it does sound like the Tesla's road detection is based on a visual algorithm rather than something more obvious like radar, and certainly both when you are too far from this sort of truck it can be nondistinct (as the road curves too much for you to see its wheels) and if you get too close it probably also becomes nondistinct (the closer you get the larger it is in your digital field)...
Please understand that the vision system in a Tesla isn't like your vision system. There is no AI which is constructing a model of a 3D world out of 2D visual data, with a road surface and 3D objects located within it. There is no human or higher-mammal level of comprehension of the scene. There are probably a series of algorithmic tricks that enable the car to determine in which direction the distant road is. The computer can then meld that information with the other shorter ranged sensors in the car that do return distance data.
The reason why things like LIDAR are used in self-driving cars, is that these systems can numerically build a model of the 3D scene without having to have an AI reconstruct a 3D scene out of 2D camera data. They return distance information, so the data starts out as 3D, so far less interpretation is necessary. In all likelihood, nothing in a Tesla understands what a truck trailer is, so how is it going to interpret that set of 2D optical data as an object that's like a moveable wall suspended a few feet in the air? There's probably only a rudimentary notion of obstacle in the software.
1. http://9to5google.com/2015/10/16/elon-musk-says-that-the-lid...
This technology was used on ships in fog back in 2004, but now that it's down to hand-held size, it seems to be more of a military thing.
There are lots of interesting things you can do with LIDAR that the Velodyne people don't do. "First and last", for example. But enough for tonight.
[1] http://www.sensorsinc.com/applications/military/laser-range-...
Although, LIDARs and sensors that do the same thing as LIDAR are getting cheaper. One thing that might change the game is the development of sensors that don't require mechanical scanners. DARPA recently demonstrated a non-mechanical way to scan a laser beam very fast and mm-wave radar is starting to approach the capabilities of LIDAR[1].
[0]http://www.businesswire.com/news/home/20131014006233/en/Pana... [1]http://www.businesswire.com/news/home/20131014006233/en/Pana...
'computer tricks' are already here with full 3d reconstruction in real time
[1] Unfortunately that is a thing. http://lanesplitter.jalopnik.com/police-hunting-sadistic-bas...
To do that, it has to understand the truck as a light colored rectangular prism with corrugated metal sides, suspended a few feet off the road surface by other structures. (wheels) I don't mean that the Tesla has to understand trucks and interstate trucking. I just mean that it understands it's a certain kind of object that's an obstruction. Doing this from an image isn't trivial. That's why LIDAR is so often used.
Here's a long theory talk by Mobileye's CTO and co-founder.[3]
[1] https://www.youtube.com/watch?v=HXpiyLUEOOY
Here is an actual dataset [1] of stills from Daimler that's used to train algorithms for pedestrian detection.
The workings of this system are similar to the cameras that are mounted above intersections to detect waiting cars (instead of sensors embedded into the roadway) [2]
[1] http://www.gavrila.net/Datasets/Daimler_Pedestrian_Benchmark...
Again, very different systems, but might be relevant when considering if straight contrast is enough to work with or not.
Why does this system allow the user to exceed the posted limit? The navigation system will likely know the posted limits.
To some extent, I agree--uncontrolled intersections on highways are dangerous.
On the other hand, in this particular case, a 14 foot tall, 75 foot long, 40-ton obstacle blocking the roadway is probably something any autopilot should be able to detect and attempt to avoid. This was not a case of the Tesla not being able to stop in time because someone pulled out in front of them. The Tesla did not slow down at all from its cruising speed of 74mph before impacting the trailer. It simply did not detect the obstacle.
I think that semi-automatic systems like this are fundamentally broken due to unrealistic expectations in the man-machine interface.
There is virtually no chance that the driver will be alert and able to detect and correct problems after hours of uneventful driving, including previous driving.
Even if the auto assist feature signals that it is confused, there is probably very little the driver can do until it's too late.
One example is the AF 443 crash where the auto pilot disengaged due to a significant but not immediately threatening mechanical problem, and the pilot (flying) got so distressed that instead of following the check lists (memory item even) he more or less just dropped out of the sky in a massive stall that took about 2 minutes.
I digress but a speculation is that in that particular case, the pilot was not helped by the fact that the airplane changed the input mode (or "law") as the sensor data was not complete due to same problem - the pilot's input was nominally consistent with the normal flight law but was in reality inducing a fatal stall.
I can imagine a system that does not allow autonomous driving on roads that have not been surveyed in, say, the last 24 hours by either another sensor equipped vehicle or perhaps a drone. When planning a route, the autonomous driver would either plan around a road that had not been surveyed within that time frame or plan a place to stop and ask the driver to take over well ahead of time.
Yet Tesla's autopilot apparently drove 74 on this road? Not that I think that speeding was the cause here (i.e. going 65 would not have prevented probably) but I think there should be a special sort of fine for speeding autopilots, that's just not acceptable at all..
people expect to use these things to travel hundreds of miles in a reasonable amount of time. there are literally thousands of these highways, and probably hundreds of thousands of crossings.
An interesting tidbit from wikipedia: "[highway] is not an equivalent term to Controlled-access highway, or a translation for autobahn, autoroute, etc."
Even low hanging fruit sometimes doesn't get fixed in a timely manner. E.g. here's an example: http://www.oregonlive.com/commuting/index.ssf/2014/11/odot_s...
I-5 is the primary North-South highway for the US west coast. In Oregon until recently there were numerous stretches without anything separating the traffic other than a grass median. Many cars have crossed the median at high speed, sometimes resulting in fatal crashes.
That particular improvement cost $7 million. But, to riff on a comment from the late Senator Dirksen, "$7 million here, $7 million there, pretty soon, you're talking real money."
Don't you have to be driving like a bit of an asshat to feel like 30 feet of earthen berm is not enough between your lane and the oncoming lane? That's a pretty big barrier and the road in question is straight and level.
Looking at the photo of the car, the 'main body' must not mean what it implies to laypersons. Or perhaps 'generally' is a broader spectrum of conditions that I'd guess.
The exterior bodywork is pretty dinged up, but even the driver side door still looks fairly smooth, and everything is still in generally the right place and looks like it may still be structurally sound at first glance.
...minus the roof, of course.