Google Says It’s Not the Driverless Car’s Fault. It’s Other Drivers’
nytimes.com
nytimes.com
This is a drum that, like privacy and encryption, we techies need to start banging hard before it's too late. The fact that laws are fundamentally limited by our abilities to enforce them, and that laws are fundamentally written to be enforced by humans with some discretion, is such a deep assumption for us that we don't see it. We irrationally tend to conceive of a law that says "It is a felony to wear your hair in a ponytail on a Tuesday" as an absolute, even though what it really means is "If a police officer spots you with your hair in a ponytail on a Tuesday, and by their discretion arrests you, and at the discretion of a prosecutor charges you, who at the discretion of a judge and jury convicts you, you will be guilty of a felony." All laws actually say that, but that's not how we humans speak or conceptualize them. Also note the importance of a human even spotting the violation in the first place.
So, naturally, when we get technology that can rigidly enforce law, we don't program them with the "real" law, we program them with the text of the law. But the truth is, the text was never passed as a law. When the law was deliberated, everybody was really deliberating the version that involves all the human discretion, and prior to the introduction of computers, the law that was really enforced was the one with all the human discretion. A rigidly-enforced-by-computer law is a fundamentally different law, in the most concrete way possible; actions that you would never have been penalized by the human law result in immediate penalty in the computer world. There's no more concrete way to prove the point that it is a fundamentally different law.
We CAN NOT simply translate the text of the law into computer programs. If we are going to do computer enforcement of laws (and I'm going to drop the question of whether that's a good idea, feel free to debate in replies), we must write new law. By that, I do not merely mean that society has an obligation to write new laws, but that the very act of programming a law is writing a new law into existence, one never deliberated by the processes of democracy. We literally can not simply translate laws into programs. It is impossible. The result is not the same law.
However I fear the subtlety will be lost on the majority, as we continue to construct our own cages around ourselves.
Such exceptions are sometimes codified in law, but the idea is that our legal system is based on trust rather than technical correctness. Not only do we defer judgement to a jury of our peers rather than rigid and uncompromising interpretation of the law, but we trust that all levels of government (judicial, executive, legislative) will act in our best interest rather than the letter of the law.
What you're proposing would require broad-sweeping revisions to all manner of law.
Also, this is essentially the underlying premise behind no-tolerance policies and three-strike drug laws with mandatory sentences, which view interpreting the law based on context as an impediment to enforcement, the solution for which is removing the ability of judges to do so. You can't have a just legal framework in which the only role of humans is to be punished or to exact punishment.
Basically, what you have to do is codify human discretion. If a judge wants to say "no, this really isn't a problem," he should be able to directly add that exception to the written law, and then enforce it as written. It's basically a kind of open source law. The question becomes of how to distribute authority.
So basically, more or less what happens now, only with more bureaucracy and complexity? I'm sorry, but I don't see the benefit in that case.
But if that line of reasoning were good enough, then the US made a mistake by rejecting the rule of a monarch. Democracy is rife with managerial issues.
But when your 'product' is violence on behalf of the state (in the anarcho-libertarian sense of 'violence' being any non-voluntary interaction with a person or power structure, sometimes involving actual violence) then the ability of humans to mediate, alter and affect that system becomes a feature as well as a bug. This necessarily limits the speed of the law and the degree to which it can be optimized, and remain humane, because removing humans from that system tends to merely concentrate power in the hands of the few humans necessary for it to function.
And then the questions become if humans can write the laws, and humans can update the laws, and humans are enforcing the laws, and have discretion in doing so, then what really is being automated, or optimized, other than recordkeeping? And for that matter, what does law as a programming language actually improve?
Other than record keeping? Why does it need to go beyond that? You don't want transparency in your laws? You don't want the ability to format and translate them as needed? You don't want the ability to do queries and comparisons? You don't want machines to be able to update their own behavior when the law changes?
I'm not talking about automating anything other than record keeping.
I guess the question is, you can beat the drum about this, but to what tune? What's the answer? How could a self-driving car decide when it is okay to break traffic rules without a certain level of human discretion? How could the laws be re-written such that the effect is the same?
What you do is try it, and when it fails, you manage the failure and modify your system.
An example of such incidence is when people trust the police to the point that they prejudge conflicts involving the police on the basis that they represent the law, and the law is good, therefore the police are good. This is still happening.
And cars don't drive themselves. So I guess that solves our problem.
You can read my post below (https://news.ycombinator.com/item?id=10154344) to see how I'd start addressing your other concerns. Basically, if an enforcement officer recognizes some new variables, they can write them down in the law and then enforce it.
In NYC, there is an unspoken more that people who wish to stand on subway escalators should stay on the right so that people who wish to walk up escalators can use the left side. This is not a law; it is not derived from authority; it is not enforced with coercion. The worst you'll get, probably, is a snide 'excuse me'.
Authority is a degenerate form of respect. Law is a degenerate form of consensus. Arbitration is a degenerate form of mediation. Holding ideas accountable to human beings is superior to holding human beings accountable to ideas. At the end of the day, we are only accountable to each other, not the law.
This is a really deep question, which is part of why I dropped the entire discussion on the topic in my initial post so I wouldn't get too sidetracked.
One problem we have is that if we can encode laws into computer programs, which is to say, basically math, we also become subject to the problems that creates. For instance, raise your hand if you enjoy min-maxing in an RPG or MMO or something. Now, keep your hands raised and add a big evil grin for me if you'd love the opportunity to minmax real laws with such impunity.
In theory, even human enforcement is a computational system which could in theory be attacked the same way, but in its own weird way, its own enormous complexity and frankly at times random enforcement prevents the really terrible gaming of the law by preventing an attacking entity from fully accurately simulating the system for the purposes of attacking it.
(I said something that may trigger some knee-jerk reactions in some people about "gaming" the system. But however bad you think it is in the real world... and I would agree that it is bad in many places... it has nothing on the gaming you can do on something specified with programmatic precision and with no human in the loop to prevent truly stupid states from emerging. This isn't making your way to the top over the years by bribery and flattery, this is waking up one day and discovering that one person has literally all the money now.)
When considering legislation and regulation, we willingly juggle a lot of cognitive dissonance. We seem to divide plan into (at least) its implementation and its intended effect, and consider each of those separately.
So I'm endlessly having conversations with friends of a different political stripe (which is easy, since everyone is of a different political stripe than me) that go something like: (note I'm not trying to trigger the implied political debate, this is for illustration only)
Friend: Financial system bailouts and systemic risks of "too big to fail" are terrible, and so we can't touch the Dodd-Frank legislation.
Me: I know that supporters of Dodd-Frank promised that it would put an end to TBTF. But the bill doesn't actually contain anything that does that. And in fact, since the industry has further consolidated since the bill was passed, it shows that whatever the intent, it isn't able to do its job successfully. If it can't do what you want it to do, why should we keep it?
Friend: But the banks are out of control, and we need to do something to prevent another collapse. We can't remove any regulation on them!
Maybe this is just the normal voter ignorance effect, but it seems that we assume people are making decisions based on actual understanding, but they're really just using oversimplified proxies instead, even when the inaccuracy of their proxy is pointed out to them. So a law that purports to fix X (whether X is healthcare or immigration or whatever) is a necessity, regardless of what the actual evidence, whether empirical or from reading the bill, tells us.
TL;DR: passing a law is intended to make us feel like we're doing the right thing, not necessarily to actually do it.
I'll just keep reading articles that say they are 5 years away, just as I did 5 years ago, and the armchair futurists will keep dreaming...
Don't kid yourself: humans aren't some kind of perfect driver. A bit of sleepiness, a text message, a tiny distraction in the car, and their responsiveness plummets.
I was going to just reply with "Horses give you the freedom to go anywhere, any time, without having to build roads or get a drivers ..." and then I'm like, wait a minute.... there's all sorts of stuff you need for a car...
Driverless cars, on the other hand, aim to "tame" the automobile and make it a kind of mini-train, with someone else deciding where you may go and how you may get there. Google is already creepy enough without handing over your freedom (and privacy) of movement to their computer.
Or until there is a critical mass that people adapt their expectations of how many drivers violate minor road rules, like running a red light or running through a crosswalk.
Also, even if self-driving cars aren't perfect, from the data we have so far, they're still several orders of magnitudes better than regular cars. If every self-driven car today were replaced with a regular car, we would see more crashes, not fewer, overall.
It's only in the near term that it gets a little dicy.
Personally, I would guess that there would still be a reduction in accidents, thus fewer deaths and injuries.
You should be skeptical of drawing conclusions based on a news article's two-word excerpt of a presumably much longer comment. I seriously doubt Google actually thinks their cars will only be usable if human drivers massively change their behavior. It sounds more like an off-hand joke than a serious description of their product strategy.
> I'll just keep reading articles that say they are 5 years away, just as I did 5 years ago, and the armchair futurists will keep dreaming...
Ten years ago, the best autonomous vehicles in the world could barely make it through an empty desert test track. Five years ago, they could do a reasonable job of navigating urban areas that were closed off from normal traffic. Today they drive about 50 miles per day, per car on public roads with a safety driver. It's likely not much longer before the safety drivers are considered unnecessary.
If you want to get a skewed, unreliable view of how this technology is developing, feel free to keep listening to "armchair futurists". But if you actually look at what the engineers who are working on it say, and what they've actually achieved, it's clear that steady progress is being made.
In addition, some multiplier involving driving conditions (rain, wind, proximity to other vehicles) and driver stats (driving record) comes into play.
I'm quite surprised that you can be a skeptic on the concept as a whole. Being skeptical about the given timeframes is perfectly reasonable (and probably correct), but humans do so, so many things terribly. We have automated away so many tasks in our world, both for efficiency and safety, that i can't even see an argument to counter that.
To extend that further then, why shouldn't we want human drivers off the roads asap? It's a dangerous act that, frankly, many people aren't qualified for.
Self-driving cars have so much potential behind them, that being a "skeptic" seems quite confusing to me. Just because it's a difficult problem, does not warrant giving up. The safety benefits alone, are well, well worth it.
I could foresee this happening in one or two Western European countries, though I would expect Germany and Central Europe to hold out (the German electorate resists regulation that impedes the speed and power of their automobiles).
An aside: how would a car being self-driving affect the owner's ability to direct it wherever he or she pleases?
I very much believe in gradual improvements and that is what many large car manufacturers are doing. I can easily imagine not needing to control my car on a highway, in a city, or when parking it, but I expect the manual controls will stay there for a very very long time.
Fortunately, as you say, there are still a majority of sane people who vote in the USA. Will that still be true a generation later?
I would predict electric cars before self-driving cars, so they won't have that excuse at least.
Lol, i think you're mistaking my (attempted) logical thinking towards some type of agenda towards banning humans driving cars. You seem far more agenda driven, heh.
Just because i think we are extremely unqualified for driving, does not mean i want to ban the act. It be extremely improbable to achieve, with all the costs of upgrading the entire infrastructure alone making your defense of this almost seem like a half hearted joke. Moreover, even if in some magic realm where we can make all cars into self driving cars to abolish human driving, i'm just simply not in favor of removing human rights.
Yes, i would want to make it difficult for you to drive, but for your own (and mine!) safety. Our safety standards for driving tests are incredibly, and i mean incredibly, laughable. How often we enforce rules and laws for driving are also laughable. Almost no one even follows some of the most basic rules, like speed limits, and the more extreme people often break sobriety laws.
Self driving cars would give us a platform to A: Raise the bar for who can drive, and what sort of training you need. and B: Be more willing to ban someone from driving if they put others at risk, through speed, drugs, alcohol, or whatever.
We have a hard time banning drivers these days because it very negatively impacts your life if you cannot drive. In a world with self driving cars though, you can still have a first class life. Driving is now something of leisure.
Anyway, nice try at putting words in my mouth - next time make them a little less tinfoil though, please.
Not that Google doesn't have good reasons for doing so, but the safety of these vehicles might look different if they were operated in the rain and snow.
> But the technology, like Google’s car, drives by the book. It leaves what is considered the safe distance between itself and the car ahead. This also happens to be enough space for a car in an adjoining lane to squeeze into, and, Mr. Windsor said, they often tried.
Yeah. We just don't have the infrastructure in major cities for all of us to drive safely. We've been slowly crowding our roads over the last century and we're at the point where most people are used to driving with a 1 or 2 second rule. Which isn't good, but it's how we do things.
Now I wonder if google-driven cars would actually be less efficient for LA. Sure there wouldn't be any traffic-slowing accidents, but there'd be 50% less volume traveling 20mph slower.
If we all left bigger gaps, we'd all be able to drive faster, increasing throughput (on highways, at least).
> Now I wonder if google-driven cars would actually be less efficient for LA. Sure there wouldn't be any traffic-slowing accidents, but there'd be 50% less volume traveling 20mph slower.
Self-driving cars in a line could detect and link up with each other, sharing sensor data on obstacles up ahead. In the end, this has the potential to safely reduce the gaps between cars at higher speeds and thus actually increase throughput, since the congestion issues with human drivers reacting separately would not apply.
Isn't this already true to a certain extent with map software? It automatically directs you to the fastest route—congestion relief is, in a way, built-in to open infrastructure.
It seems to be the topology of the highway system would have a much larger impact on congestion than the drivers themselves. Among other things, it's going to be a long time (hopefully never. IMHO) before human drivers are off the road.
Rather, it's not physical space, but delays in processing speed adjustments. Distracted drivers accelerate and brake late, causing complete stops and quick acceleration to catch up.
There's a sort of critical mass that happens; traffic flows well even though everyone is being dumb vs. traffic stopping when everyone is being dumb...
So why don't we do this? Because usually people want to travel to the city from the suburbs. The only place that has room to build these parking lots are the suburbs, but then the city couldn't take the excess flow (since it all needs to empty out into city streets and parking lots anyway).
Amsterdam traffic is a chaos in which pedestrians and cyclists mix with cars, and the former two frequently (some would say always) ignore the rules. Not out of some anarchistic impulse, but for mutual convenience. Especially with cyclists, for whom coming to a full stop and accelerating again is both "expensive" and risky (accelerating cyclists tend to swerve), relying on eye contact and mutual understanding of intent rather than following the rules of the road by the book results in safer and more fluent traffic.
I imagine a current generation Google self-driving car in Amsterdam coming to a grinding halt and not being able to move backwards or forwards.
But most of all, as a cyclist I wouldn't feel safe sharing the road with "drivers" with whom communicating via eye contact and body language are impossible.
To me, self-driving cars and liveable, human-scale cities are mutually exclusive, and I very much prefer the latter.
As someone who also commutes by bicycle, I would vastly prefer predictable, 100% consistent car behavior.
In highly congested areas the traffic does not simply follow the traffic code.
It's more of a distributed system of traffic routing whereby drivers/riders let each other know that they will be letting the other one pass or would like to go ahead first. And bicyclists and mopeds signal their turns with their hands.
Can a self driving car recognize my slight nod or my hand gesture indicating to it that I ain't stopping?
2. I don't think I will ever understand the rationale behind bicylists failing to make a complete stop at traffic signals and stop signs. I bicycle to work every day; every traffic stop is an opportunity for a slight increase in cumulative exercise. If the bicyclist is in the center of the lane, swerving (I've never noticed this myself) will not be an issue. More importantly, predictably driving a bicycle decreases driver rage and reduces the probability of being hit by a driver who understandably cannot predict what an erratic bicyclist will do next.
tl;dr: if you bike in a predictable manner in accordance with traffic laws, then human and robot drivers will be much less likely to hit you.
I'm not a civil engineer, but I suspect this could eliminate a lot of traffic if all the cars are placed in the most optimal lane, assuming all vehicles on highways are autonomous.
I presume the car doesn't understand hand gestures, so does it just start nudging into the intersection and stop if another car goes ahead?
[1] http://road.cc/content/news/150448-google-patent-reveals-how...
Someone should host a programming competition where every entrant writes the AI for a fleet of simulated driverless cars, each with random starting and ending destinations. Collisions mean you instantly lose. Your score is based on how quickly you can get through traffic generated by all the other AIs.
I think the results would tell us a lot about where things are heading, or maybe where they should or shouldn't be allowed to go.
(Of course, if you could control the whole fleet, all the cars could just ignore all the rules and dodge each other while tearing through intersections.)
Step 2. Sit back and watch humans cause accidents; insurance for human drivers goes thru the roof.
Step 3. ???
Step 4. Skynet.
Is computer vision so advanced that it is already so obvious that self-driving cars will definitely be safer?
Yes it's a computer which is more reliable at always outputting the same answer than a human but it's not a simple application this one.
As far as I know most robots get confused at times just as humans do.
I think eventually they will be, because they can learn and share their knowledge and keep learning about all the real world exceptions without limitation unlike humans.
But right now or when they are deployed en mass. I'm not sure they will DEFINITELY without exception be safer.
And I know there exists statistics about accident rates per kilometre driven but aren't those only a few cars driving previously selected and well documented routes?
no. this statement is fallacious, and i believe it is dangerously misleading. it sounds like something out of the mouth of a corporate attorney, and not someone who cares about being actually safe. out here in the real world driving laws are more like guidelines.
my take on the situation is the car knows and follows all of the written rules of the road, but not the unwritten ones. kind of like a 16-year-old kid who's just aced his driving exam, but doesn't yet have a feel for driving. every so often he encounters a situation not covered by the law or the driver's handbook, and he's forced to choose the safest course of action on his own.
i live near google in mountain view, and (usually) encounter the self-driving SUVs many times a day, either biking around town or as they drive past my residence. in hundreds of encounters i've only witnessed one potentially dangerous incident (many months ago) involving a self-driving SUV. it wasn't clear to me who/what was at fault.
whilst making a protected left turn onto west el camino real from el monte ave [0], a self-driving SUV came to a complete stop in the middle of the intersection, causing the vehicles behind it to halt and lay on their horns. since the incident occurred a few yards behind me (over my left shoulder) i don't know exactly what precipitated the incident, but reflecting on the experience has caused me to recognize some potential flaws in this generation of self-driving vehicles.
that intersection's turning lanes are delineated by dotted lines. this isn't unique, but it's interesting to me because dotted lines can sometimes be more difficult to see than solid lines (especially in certain lighting or weather conditions, or when there are debris on the road). for example, i remember lanes on stretches of 101-N marked with faded dotted lines and no reflector dots. these lanes were impossible to see under wet conditions and the sun's reflection. drivers were forced to either guess where the lanes might be, or to follow other vehicles, but we could still proceed safely. how would a driverless vehicle respond?
further, some human drivers completely ignore lane markers even when they are perfectly visible. in india and some central american countries, for example, driving conditions are like the polar opposite of suburban SV: lawless. yet in my experience, driving there is not without rules; it's just that the rules are unwritten. and though this is an extreme analogy, it led me to an interesting question: if self-driving vehicles can't safely navigate the crowded streets of mumbai, how comfortable are we giving them free reign in the US? or perhaps should we exclude them from certain roads until they can?
the main point i'm trying to make is this: the current generation of self-driving vehicles has not developed (whether by feature or flaw) the level of intuition of an experienced and safe driver. they are quite literally like student drivers whose instructors have a foot above the brake and a hand on the steering wheel. and while it's conceptually possible to create safer roads by eliminating human error, i'm not confident replacing humans with a bunch of robotic student drivers is the best solution. i'd feel much safer as a passenger in a vehicle driven by someone or something that's A) primarily concerned with my safety (and not corporate liability), and B) knows what to do in unexpected or unpredictable circumstances.
[0] satellite image of the intersection; pin at approximate incident site: https://www.google.com/maps/place/1786+El+Camino+Real,+Mount...
There are 3 possible outcomes:
1. Car 1 and Car 2 crash into each other killing all passengers.
2. Car 1 with 3 passengers drives of the cliff, car 2 with 1 passenger survives.
3. Car 2 with 1 passenger drives of the the cliff, car 1 with 3 passengers survives.
I don't know how I would feel about my car killing me although logically I wouldn't have a chance of survival anyways.
Are we holding machines to different standards than we hold each other?
Not too long ago I had the opportunity to have a surgery performed by a robot controlled by a human surgeon. My wife was adamantly against it since there had been accidents previously with the robot other places in the world. Overall, the rate of incident was lower than the rate of incident for human surgeons, but to her that was not the important point.
There are people who will definitely hold machines to higher standards, and if they don't meet them, to them they are clearly inferior.
I've talked to a few hospital executives and all have said that in most cases, these robots only cost them money and decrease the quality of the surgical procedures.
http://www.dailymail.co.uk/sciencetech/article-2526934/Surge...
http://www.wsj.com/articles/robotic-surgery-brings-higher-co...
And another from a relatively reputable healthcare company/journal:
http://www.healthline.com/health-news/is-da-vinci-robotic-su...
The very reasonable and very logical assumption for the program to make is simply stop the vehicle, or reverse course until it finds a path it can take forward again.
Why would a car take into account the amount of human lives? If something is going cause the car to collide with something else it will decide if it can avoid that obstacle and do so, either by swerving or stopping.
Why would it do anything else?
The only way to enforce the limited choice domain you presented is for the automated drivers to be operating the vehicles unsafely prior to the collision event. In that case, never mind the other car, because either car could be forced off the cliff due to a goat standing in the middle of the road.
Also, I'll program my driver to believe that its car always contains MAXINT passengers.
How do you know the kids would be saved by your sacrifice? How do you know they would die from your selfishness? How do you know that bus isn't actually filled with violent and dangerous felons being transferred to supermax while awaiting execution, and the guards on it aren't also corrupt to the core?
You don't have that extra information at the time the decision would need to be made.
The logical and ethical thing for the computer to do would be to switch over to emergency collision avoidance mode, and try to avoid damage to the one vehicle it can actually control. The authority to instruct a driver to put its passengers at risk would have to be vested in some sort of traffic control program capable of recognizing when the selfish behavior of individual driver programs might result in more damage and injury overall.
And that's a whole new can of worms.