Metro Crash May Exemplify Paradox of Human-Machine Interaction | Hacker News Reader

Metro Crash May Exemplify Paradox of Human-Machine Interaction

washingtonpost.com

34 pointsrobg30 comments

One of the reasons why highways and thruways are relatively safe is because of an enforced social contract (e.g. if you speed, you may get ticketed and if you drive erratically, you may put your life and other people's lives in danger)

With software, there is no such powerful peer contract. Software is not yet as good at self-healing (e.g. braking suddenly) if something unexpected happens. How do you handle a system exception safely and robustly at 65mph?

Automated cars are a pipe dream. Automated auto-pilot for cars on highways (e.g. automated platoons of cars moving in lock-step until they reach their exit) a.k.a Intelligent Transportation Systems (ITS) will be persistently hobbled in adoption because failure of the interface between humans and machines is dangerous at highway speeds. Recent ITS research has focused on trying to academically prove MITM attacks cannot work for certain ITS systems. Also, most people like freedom to drive whenever, whereever they want.

Highways are "relatively" safe? Relative to juggling chainsaws, maybe. After disease, cars are the leading cause of death in just about any country wealthy enough to have cars.

Drivers need all the help they can get from automation, even if it backfires now and then. The fact that any idiot who can spell his own name is allowed to hurtle through crowded streets with a two ton killing machine and no failsafes of any kind is clearly absurd.

After disease, cars are the leading cause of death in just about any country wealthy enough to have cars

No, they're far down the list. Less than 2% of all deaths are from auto accidents, fewer than influenza and pneumonia and 20x less than cardiovascular diseases.

http://www.the-eggman.com/writings/death_stats.html

2% is exactly the proportion I was trying to convey and it's plenty to make my point, which is that an automated system would surely be scrapped for killing half that many people, even though it would be saving millions of lives.

Systems that become safer by scaling back automation are anomalies. In general, efforts are better spent improving the automation.

The man vs machine dichotomy is contrived anyway. We personify and blame machines as a scapegoat. I prefer to think of automated systems simply as tools, extensions of human capability. Tools don't screw up, we screw up making them or using them.

automobile accidents is the first non-disease on the list, so i think the original claim is correct, even if the lack of statistics blow it out of proportion.

IMHO, people underestimate the risk of death and injury from driving. while inciting fear is not really a good educational tactic, i can understand the frustration. my response is to wear a helmet whenever i go for long highway rides. and to avoid transportation as much as possible. (*obviously not for everyone)

edit - disease effects a different crowd than motor vehicles, which i think is part of the risk/cost evaluation.

Everything above auto-accidents on your list is a disease, so technically he's right =P

Oops, let me rephrase.

While car accidents are the leading non-accident way to die, you're 45x more likely to die from a disease.

I remember seeing some statistics on traffic accidents from a swedish government department. And here it is (sadly all in Swedish):

http://www.socialstyrelsen.se/NR/rdonlyres/2B5A6B25-2026-470...

Page 51-51 has graphs of fatalities ages 1-15 years.

Page 71-72 has graphs of fatalities ages 16-24 years.

Page 117 has graphs of fatalities ages 25-44 and 45-64 years.

The green area labelled "Fordonsolyckor" are those that are traffic accidents.

Summary: At young ages, traffic accidents are a large part of all deaths, but after 45 years of age, the reason of death is dominated by cardio-vascular diseases.

Also, most people like freedom to drive whenever, whereever they want.

Count me in that group. I rarely ever use cruise control (even on long trips) because I've noticed the lack-of-focus effect. I also drive a stick and when I had a rental car it became quickly apparent to me why people can lose focus when driving a slushbox sedan - it's incredibly boring driving an automatic.

With respect to cars, I'm a big believer that technology and systems should do everything they can to aid the driver without interfering or taking over the core human task of driving the vehicle. The Nissan GTR is a great example of a cutting edge car that enables a human to rip around a track faster simply because systems are faster than humans when it comes to managing braking, shifting, steering correction, etc.

F1 cars are the same way; they're incredible combinations of technology and computer systems, but in no way will a computer ever take the place of Michael Schumacher.

With respect to cars, I'm a big believer that technology and systems should do everything they can to aid the driver without interfering or taking over the core human task of driving the vehicle.

That's a difficult line to draw, especially when (as you describe with the GTR) the job of the computer is to react faster than the human. For 95% of my car trips, I would feel plenty "in control" if I could tell my car where I wanted to end up and let it take me there however it thought was best, assuming it got me there reasonably quickly and safely. After all, if my car was driven by a human chauffeur, nobody would question whether I was in control. Why would an equally reliable computerized control system present a greater challenge to my feeling of control than a human chauffeur?

The distinction is clearer than you think. The technology in the GTR and F1 cars is limited to "managing braking, shifting, steering, etc." Intent is still left entirely to the human driver. He decides when to brake, the computer helps to do it as efficiently as possible.

It's easy to say where the line is right now, but you imply that at some point "intent" might be usurped by the computer. I don't think there's any clear way to define that limit. If the driver merely tells the computer where to go ("the 7/11 on 4th Street") and the car does the rest, does that cross the line? Does that usurp the "core human task of driving?" You are ready to let computers calculate the most efficient way to brake, and I am ready to let them do much more, as soon as the technology is available and reliable. Where is there a clear line in between?

Well it seems clear to me that the "core human task of driving" is the supplying of inputs to the 3 major controls: brake, accelerator and steering wheel (5 if we include clutch and gear selector). I'm not necessarily agreeing with kirse, just saying that the line is well defined.

Adaptive cruise control (and even regular cruise control) definitely crosses the line and is available today in a bunch of mass market vehicles.

The metro crash may also exemplify an ATO system with a single point of failure, or poor maintenance, or poor training, or many other things. I am disappointed that no articles I've read about this crash have discussed the actual systems beyond them containing track circuits.

So does the Air France crash. Speaking of which -- could GPS be used as yet another backup device for airspeed? It might have high latency, but it would be better than nothing.

No, the GPS is not a reliable backup device. The issue is not latency. Aircraft GPS systems generally update faster than 1 Hz, which is plenty fast for most things. The real issue is that GPS gives you the ground speed, and not the airspeed. The wind speeds at 30,000 ft can exceed 200 knots [1]. Therefore, depending on the wind speeds and direction of travel, your airspeed and ground speed can be vastly different. Of course, it is theoretically possible to correct the GPS groundspeed using weather data (would have to be downloaded from external sources) to correct for upper winds, but this is not reliable over much of the surface of the world.

[1] http://en.wikipedia.org/wiki/Jet_stream

This is all true, but except for stall avoidance, when navigation or landing you care about ground speed. The old ILS doesn't know airspeed or ground speed.

GPS coupled with a laser gyroscope can provide data at KHz frequencies.

It is better than nothing, but the speed through the air is the most critical parameter for an airplane, and GPS measures speed over the ground. If a sensor goes out, the computer could assume that the wind is constant, and display an estimated airspeed, but the probe in this case iced over, in this case because of a thunderstorm. The wind is all over the place in a thunderstorm, and it wouldn't be very useful.

GPS is used for all these things. It's even used for automated landing systems: http://www.freeflightsystems.com/gps_landing.htm

I think the intentional lack of resolution in civilian GPS is in part because it is not intended to be an air navigation back-up system.

I was under the impression that the "degraded civilian" properties of GPS have been disabled for some years now.

I believe that IFR-certified GPS systems have the added capability of factoring Differential GPS. This is an additional GPS beacon (aside from the satellites) originating from the airport you're landing at, which have a location that's very accurately known.

High-price civilian GPS receivers are capable of this -- except you rarely have access to such a beacon. I mean, the restaurant I'm going to eat at isn't going to provide my little TomTom with a differential signal. With aviation, it's important enough that high-traffic airports provide the DGPS beacons.

This isn't true. IFR certified GPS units generally don't have Differential receivers. The main reason for this as far as I can tell is integrity. To be IFR certified, the receiver has to be able to tell when a satellite is giving bad data. In the older units, this was done with RAIM, which basically compares what is coming in from all of the satellites, and determines if all of the satellites are providing positioning that is relatively consistent. DGPS is a relatively simple system, and there's no way to check the DGPS signal for integrity. Without the ability to check for integrity, the additional accuracy is useless, as the design of instrument approaches is based on worst-case scenarios.

Now newer IFR certified units do have WAAS, which is similar to DGPS, except the signal is transmitted from a satellite, and is based on measurements from a couple dozen ground stations, the correction interpolated in the unit.

There has been a proposal to implement a system called LAAS for a number of years, which would be like DGPS, but would have the integrity checks that are required for aviation. However, it never really got off the ground.

As it stands, with WAAS, you can be guided to 200 feet above the runway, but this is not as good as the old analog radio ILS, which, if setup in a sophisticated way, can take a plane all the way to the surface.

Yes, but I believe they still reserve the right to restore them whenever they choose.

From Wikipedia (http://en.wikipedia.org/wiki/GPS#Selective_availability):

On 19 September 2007, the United States Department of Defense announced that future GPS III satellites will not be capable of implementing SA,[59] eventually making the policy permanent.[60]

Thanks, I did not know that.

Aircraft with Wide Area Augmentation System (WAAS) are certified to use GPS as a primary navigation source. [1] [2]

[1] http://www8.garmin.com/support/faqs/faq.jsp?faq=221

[2] http://en.wikipedia.org/wiki/Wide_Area_Augmentation_System

On the topic of "The better you make the automation, the more difficult it is to guard against these catastrophic failures in the future, because the automation becomes more and more powerful, and you rely on it more and more." it's instructive to read the last story of Asimov's "I, Robot" called the "Evitable Conflict" in which the machines lead the Earth's progress.

This touches a lot on why the UI of your application is crucial.

Reply on news.ycombinator.com