What If Your Autonomous Car Keeps Routing You Past Krispy Kreme? | Hacker News Reader

What If Your Autonomous Car Keeps Routing You Past Krispy Kreme?

theatlantic.com

75 pointskdazzle89 comments

Just one of a bazillion questions that comes up when your car is autonomous. What happens when the sekrit tech elite want to kill you and so they make your car drive off a cliff? Interesting questions to be sure and already being examined on such shows as "CSI" in the US.

Basically one of the reasons new technology rolls out slowly is that there are lots of people asking these sorts of questions and trying to have answers before they ship. Sometimes that works well, sometimes not (cite battery issues on 787).

Another case worse than partially-hijacked trips was mentioned in a slashdot post, and if I recall correctly, there was a vague reference to the autonomous-car-makers considering it, and it has bothered me ever since.

Imagine a situation where a crash appears to the AI of car C1 to be be inevitable. But also, if C1 executes maneuver M1, its own occupants will be saved from harm, but another car C2 will have a worse crash or will crash when it otherwise would not - but if C1 does maneuver M2 instead, its own occupants will be killed or suffer worse harm, but the occupants of C2 will be better off. What if, in such a scenario, C1 would decide to "sacrifice" its own occupants to minimize the total harm (e.g., C2 has more occupants)?

Self-driving cars might well be designed this way. It might be mandated by law. Authorities would try to reassure objectors by saying the "avoidably killed by your car" outcome is very unlikely and that you're better off in the big picture of all risks.

Edit: On a closer reading I note that the linked page http://www.theatlantic.com/technology/archive/2013/10/the-et... covers, not exactly the above, but similar scenarios.

Apparently in the UK almost every inevitable crash will end with the driver swerving right, usually putting his passenger in harm (driver sits on the right). I'm told that's a natural and unavoidable reaction, so if it's true I assume the opposite holds for America and etc. I guess that's another decision the car has to make, do you hit an unavoidable obstacle head on or swerve, presumably drastically changing the survival chances of each passenger. The baseline human panic reaction is almost certainly easy to 'beat' but it's a horrifying though that a car would have to make that decision.

I've always thought that the protection part was a fallacy and the more obvious conclusion is that they swerve in the direction they are used to safely passing with.

It is likely that an AI sophisticated enough to make the consideration will be sophisticated enough to avoid the question.

That is, there isn't all that much reason for the cars to be operating on the precipice of disaster.

(If you want to say "But what about uncontrollable and unforeseen circumstances?", we should hope that we make the test out to be 'better than most humans', not 'perfect')

Assuming we change slowly to self-driving cars and they're operating for a while in an environment with human driven vehicles, no disaster scenario should be assumed out. They will have to deal with the situation that kill people right now, and them choices could easily reduce to "kill my passengers or those drunks in that car hurtling toward me" for example.

Better than humans is great, sophisticated AI is great, we can't expect perfect but I don't see a situation where they enter an environment much less dangerous than today's.

I expect the standard heuristic when the program detects an unavoidable collision will be to minimize impact energy, whatever that means.

(It won't have good information about the mass of oncoming traffic, but there aren't that many situations where choosing to collide with the oncoming vehicle makes any sense at all)

As with most thought experiments, it's a little far-fetched though. How often, in the run-up to a crash, is it really possible to make oneself safer (or less safe) at the expense (or benefit) of another car?

A more common scenario might be driverless cars that don't communicate with each other and therefore try to "optimize" a crash without knowing how other cars are going to respond – potentially far more dangerous than the magnanimous AI you talk about.

And imagine if the AI also rated the car occupants on their future value to society.

If we're talking about Google, I assume it would rank people based on the number of friends they have in Google+.

The cynical side of me suspects that the "Volvos" of the future will make this choice less often than the "Pontiacs". The rich will still have safer cars, AI style.

It seems like an interesting branch of "applied theology". Does one, in the end, have faith that these tiny entities imbued with lesser-god-like driving abilities really have one's best interests at heart?

You appear to be claiming this is a bad thing, when (assuming it worked as planned) this would make you safer, since on average you'll be in the other car.

Of course, an irrational fear of safer cars is already considered to be one of the big stumbling blocks for self-driving cars, even without this scenario.

http://en.wikipedia.org/wiki/Trolley_problem

I need to start carpooling more

Indeed. Here's a simpler one than the "sekrit tech elite" wanting to kill you...

What happens when the equivalent of the Mariposa botnet, but this time for autonomous cars, roots 12 million of autonomous cars?

But, this time, we have to believe that security is going to be done correctly, right? (this time, promised, no backdoors, no 0-days, no compromised servers, no compromised keys, etc.)

I does really hope this technology rolls out as slowly as possible because the security issues do really scare me.

oh god, ransomware for your car...

"We are going to kill 100 drivers every day until 1bn in bitcoin is sent to 1...x"

Just imagine the economic disruptions on day 3 when no one dares drive anymore

Didn't the first one already happen with Michael Hastings? Although to be fair, it's just suspicions and not any officially accepted story. Nonetheless, they're fairly strong suspicions, same as with the Karl Koch case.

Regardless of whether or not that is true (and full disclosure, I doubt that it is from what I've read) it is surprising to me how something with tenuous evidence can be presumptively believed to be true in certain circles. His own wife says it was a "tragic accident", and his brother says he believed Hastings to be in a "manic episode" at the time.

Why is it that random people on the internet have so much more suspicion and rage than his loved ones who were closest to him?

This isn't really meant to be directed at you, so much as expressing my frustration with a sort of casual canonization of a dubious event as a conspiracy.

Because the motive and the modus operandi make a good fit for the scenario.

The family is not interested in the emotional baggage of foul play, or the hassle of an arduous investigation, which would, hypothetically, probably be fruitless, even if suspicions were accurate. It's much, much easier for loved ones to swallow an official story of a tragic accident.

For total strangers, not burdened by emotions, the scenario is objectively disturbing.

I haven't been following the Hastings case, so I can only reply in general:

Following the recent Snowden disclosures, you should question yourself at disbelieving conspiracy so easily. You might be right (probably are, for all I know), but you have no reason to believe so other than pure faith.

Ask yourself this (and answer truthfully): A year ago - did you believe that the NSA is actively sabotaging standards, intercepting shipments in order to modify the software to be under their control, etc? If you didn't (and I'm sure you've had the conspiracies mentioned), then why on earth would you believe "the official word" on anything like remote control of a car?

You might have a lot of faith in the official status-quo for no reason at all.

I am fucking sick of people being surprised by the NSA revelations.

NSA malpractice has been documented all the way back to 1975 by the Church Committee's papers: http://www.aarclibrary.org/publib/contents/church/contents_c...

Then later on we had Clipper chips, ECHELON, Presidential Surveillance Program, TEMPEST, Total Information Awareness, DCSNet, Room 641A and plenty of other surveillance disclosures.

This whole stigmatization of conspiracy theorists as loons is wholly unfair, misguided and shows the speaker's ignorance of how intelligence agencies operate. Conspiracy theories aren't just idiotic pop culture ravings about Illuminati, reptilian men and ZOG.

Because conspiracies make for a good story to fill your hunger for curiosity, and when you are only loosely versed in the circumstances it is easy to find reasons to believe it was a conspiracy.

>What happens when the sekrit tech elite want to kill you and so they make your car drive off a cliff?

It's considerably easier to physically tamper with a car, such as the good old Hollywood standby of cutting brake lines.

Why worry about a fantasy of people trying to attack you remotely when there are plenty of real organizations that are much more likely to tamper with your car now?

> It's considerably easier to physically tamper with a car, such as the good old Hollywood standby of cutting brake lines.

It's only hard to exploit a serious security flaw the first time. Until they close it, it's relatively easy to exploit it thereafter.

The relative lethality seems likely to favour the software option too. Personally I doubt I'd get above 10mph before I tried using my brakes - (manoeuvring into and then out of my road.) Compared to, 'Wait until you're on the motorway and accelerate as hard as you can into oncoming traffic.' I don't see too many people walking away from the latter case - the closing speed would be something like 190mph, disregarding braking on the part of the oncoming traffic.

Because technophobia is harder when you don't have a technology to be afraid of.

Nearly exactly this happens in Stross's Halting State

You mean, bidding for a better bus route?

No, assassination by hijacking of automated cars

I had speculated about something like this with friends when it was announced that Google was getting into autonomous driving cars. The combination of G+, Maps, Zagats and a Google car that drives itself would give possible scenarios like the car doing things like asking you:

"You haven't had lunch yet today. There's a highly-rated Italian place just two minutes from here, would you me to take you there?" then afterwards, "what did you think of your meal, on a scale of one to five? I'll share it with your G+ circles."

The only thing that bothers me about this (besides the obvious data privacy issue) is how much power it gives one entity Google, to subtly manipulate nearly anything.

My phone already does this with Google Now.

It suggests where to eat lunch if you haven't had it yet? Serious question, that would be an awesome feature, but I couldn't find anything on it. I don't regularly get lunch out but mine is always giving me "Places Nearby" which seem to have nothing to do with my interests.

It's always seemed to me that places nearby is somewhat based on time, I'm more likely to get the cinema in the afternoon and a restaurant in the evening. It could just be coincidental but it'd make sense if it is influenced by time. That's not as clever as knowing I haven't had lunch yet, but since I often have it with me or buy it in a supermarket or something, they'd have no way of accurately knowing anyway.

Why is this a hypothetical future scenario? AFAIK, Google is perfectly capable of doing that even now, with route finding/guidance on Google Maps.

Right, Google was also recently granted a patent for something like this: http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=H...

I had the same thought. I imagine that, in said future scenario, the author is assuming that the ubiquity of self-driving cars has progressed to the point in which drivers are far less attentive to their routes (given the autonomy of the cars). Compare that to the present; if you're using gmaps, you're engaged with where the car is going and perhaps more likely to notice the alternative route to Krispy Kreme.

Then there's the safety aspect: current mapping/gps providers would really come under fire if they were asking you to take your eyes off the road for any longer than they already do. Most, if not all, offer enough voice guidance to remove the need to look at the screen at all. Even if the decision-making process were to be done out-loud, I could see some safety concerns arising.

I don't think anyone is going to be dumb enough to try the specific scenario this article outlines. The real world is noisy, and in practice, an objective like "reroute path to include Krispy Kreme, but only if it isn't too far out of the way" is going to have too many edge cases to do 100% reliably.

And it only takes one "my car made me 40 minutes late to work by trying to take me by a Krispy Kreme in the next town" story to turn this idea into a gigantic embarrassment for everyone involved.

Until everyone shrugs it off and gets around it by holding their meetings at krispy kreme.

> an objective like "reroute path to include Krispy Kreme, but only if it isn't too far out of the way" is going to have too many edge cases to do 100% reliably

Too many edge cases? I'm sure you would have said the same thing about self-driving cars not too long ago. In a world where cars drive themselves, choosing an alternate route based on traffic and waypoints is a relatively easy problem to solve.

It's a matter of risk vs. reward. The reward for self-driving cars is huge. If it screws up once or twice and makes you late, you don't have a whole lot to complain about. From the developer's point of view, it's not all that embarrassing because it's naturally framed in the context of "yeah, but it's a self-driving car."

But the situation is totally different with advertising. It's not "I was late because my amazing scifi technology malfunctioned"; it's "I was late because of a crummy commercial". The consumer doesn't get any tangible reward for this, so they're just pissed off. And the developer and advertiser end up embarrassed because they did something shady and it screwed someone's day up.

Honestly if autonomous cars can reduce even 10% of accidents caused by drunk driving and other human errors then they will have payed their way and more.

But speaking of paying, self driving cars also won't speed, turn right on red without a full stop, etc. Which means traffic tickets would be a thing of the past, along with the revenue they bring in.

Driving while black, failure to provide proof of ..., tail light out.

If a cop declares "your" car's driving was reckless or too fast for conditions, how exactly do you fight that in court?

You could probably fight it in court with the electronic logs that would presumably be present in a self-driving car.

With dashcam?

At the same time policing costs would decrease dramatically.

That's probably not true. I've gotten the impression that a sizable chunk of police funding comes from speeding tickets.

Paying an officer to do something low-risk like camping out next to a road for a shift isn't really expensive. Same with monitoring systems like cameras on traffic lights. Set-up cost is significant, but they probably pay for themselves within a year.

The initial cost doesn't matter. If an officer doesn't cost much, yet you don't have to pay him any more than that is a 100% decrease in cost.

Second I know that the officer is not the only cost involved. What about maintaining a fleet of top of the line vehicles? How about paying all the support personnel such as IT Staff, dispatch, managers, etc? All of the other expensive toys carried around by the chaser?

> Second I know that the officer is not the only cost involved. What about maintaining a fleet of top of the line vehicles? How about paying all the support personnel such as IT Staff, dispatch, managers, etc? All of the other expensive toys carried around by the chaser?

Arguable. It depends on how badly you want [insert autonomous car producer here] to run your local police department from the cloud.

I was thinking about this for other reasons (it is beneficial for the auto-car to know about road closures), but a reasonable implementation has the vehicle interpreting rules data provided by the governments where it is operating, so you can do certification by making sure the vehicle 'correctly' interprets a given data set.

Having the rules data provided by the government is a fairly straightforward way for the vehicles to work even after the builder repudiates maintenance (maybe that is better said as 'to continue to work longer after', but whatever).

(I'm not worried about the cloud, hand-wringing and paranoia are going to make these things at least function independently or keep them off the road altogether)

Who is responsible for paying the fine if the self-driving car breaks the law?

Currently, the driver.

Four people are in the car, two friends, and two people who are on the title and regs. There is no steering wheel. Whose license do the points go on?

Currently, that situation isn't possible, as self-driving cars still require an attentive driver who can take over at any time.

In the future? That's a hard question, and I'm sure however the legal system deals with it first will be wholly unfair and illogical. It'll be a battle between car owners, manufacturers, state and federal bodies, cities and counties, certification and safety agencies, etc.

Probably the same person that gets points when they cause an accident with their bicycle: i.e., nobody, because you don't get points for that.

Are we talking push bikes? At least in the UK you don't need a license for that anyway. I guess it's possible you wouldn't need a license for a self-driving car, but while regulations demand a fit driver who can control the vehicle that won't be the case.

My point is, I think regulations will change as technology improves. As transatlantic flights started, engines were unreliable, so regulators forced airlines to take awkward routes when crossing oceans (so that they would be 60 single-engine flying minutes away from a diversion airfield). Now that engines are more reliable, the regulations are significantly more relaxed, sometimes allowing 330 minutes to a diversion airport.

The point is, regulations are not set in stone. Companies can advance the state of the art, and ask the government to regulate less strictly.

As it stands now, self-driving cars are a research project, so we'd expect them to be regulated very strictly. As testing shows them to be safe, then we can relax the regulations. If self-driving cars without a supervising driver end up being safer than a normal car with a normal driver, it would make sense to not require a license. If an accident happens, it happens. Car accidents are nothing new.

I hope it's Google.

Actually assuming 190 million licensed drivers (although drunks kill all manner of people, including themselves) and 10K drinking related deaths per year (how many are caused by drunks as opposed to would have happened anyway with at least one driver who happens to be drunk?)

Anyway assuming all victims loose 100 years of life...

10e3/190e6 * 24 * 365 * 100 * 0.1 = a delta of 4.6 hrs/day so I think your hyperbole is pretty much mathematically correct.

Our local krispie kreme closed, so the next closest is about 100 miles away, so if a "driveby" was purchased for me, it could actually burn up 4 or 5 hours of my time, especially during rush hour.

That's too idealistic. The reduction of accidents caused by human errors could very well be offset by a substantial increase in machine/OS/SW/HW caused accidents.

But humans are really shitty drivers.

The software would have to be written by chimps for it to be worse than human drivers.

I fully expect all kinds of errors to cause human death. I just hope the backlash doesn't destroy something that could save very many lives.

That's pretty unlikely though. It's not like cars are going from manual-everything to autonomous all at once. Multiple levels of traction control and steering assistance are already in control of your car, and they prevent more accidents than they cause. Autonomy is going to be rolled out in pieces and tested little by little.

Between Glass and Google's self-driving cars, it is quite clear that the future of advertising is going to be potentially quite distopian where virtual billboards pop up based on your interest and location and where ads playing in music are based on where you are and what time of day it is. They might be as simple as a McDonalds ad for their mighty wings and your car will change course if you say "I'm loving it!".

This is actually the sort of natural endpoint of what I thought would happen with AR mobile apps and now that Google has Google Now on android, is doing glass and self driving cars, it is pretty clear what the future looks like.

It's more ads.

Pohl and Kornbluth nailed it in the 1950s: https://en.wikipedia.org/wiki/The_Space_Merchants

I am OK with this. as long as there is a web service that tells you when the "hot" light is on.

Actually, I had a paper some time ago already on the issue of combining preferences with optimization (shameless plug). http://dl.dropboxusercontent.com/u/16703681/homepage/mabrams...

Better yet, would you trade ads and impulse buys for free rides?

So you never have to own a car, but every time you go anywhere your car is actively trying to get you to spend time and money doing things you normally wouldn't do -- play online games, visit donut shops, and so on. Would that be worth never having to own a car?

That actually sounds like a brilliant angle for selling an automated car service.

"Our cars will take you wherever for a dime! To make up the difference we will be playing ads and/or having you partake in surveys during the ride."

Sounds cool if you live in a city particularly.

Google's patented this idea; see, e.g., http://arstechnica.com/gadgets/2014/01/google-patents-ad-pow...

Nope. Most of the personal benefit of not having to drive would be that I could do other things with my time. If I've got to be doing something I don't like anyway, then that defeats much of the point.

Consider that such base manipulation would be used primarily on the poor.

I think it is a very long way down the road before the poor start getting self driving cars.

If they work well they should be a more efficient use of capital than a normal car (higher utilization, better adherence to maintenance schedules, etc). That creates an opportunity for per mile charges to be cheaper than owning a car.

(they should obviously be some combination of more profitable and cheaper than a taxi service...)

Self-driving short-term rentals ala Zipcar or Car2Go?

> What if...?

Why not present the user with the option before their trip begins so that you're not wasting a lot of fuel and so on? I can't see people being too happy with you if you start routing their lives around your convenience. Advertising currently costs the end user very little - but there's certainly a transaction cost involved. If you start doing this to people, there'll be a significant incentive for them to disable the system in some way, or opt for a different one.

"Our cars only take you where you want to go." Would be a pretty good advertising message. "Average 10% faster journey times from A to B. You're in control. Take back your life." Etc.

This isn't that scary, in situations where the passenger does not own the vehicle the route is probably going to be shown to them on the device that they use to request the car, so there is plenty of opportunity for a negotiated route (rather than a route determined strictly by the vehicle/operator).

The trick is convincing enough people that they should pay attention to the route they pay for.

This already doesn't work with human powered autonomous vehicles aka taxi cab.

Maybe I'm missing something, but..

Doesn't this basically already happen? I would imagine retail space along main thoroughfares (e.g. "Main street" or space right next to a very popular highway exit) is substantially more expensive than one that sees 5 cars a day. Retailers already basically know what route you're likely to be taking - the one that most other people take.

Granted they don't physically stop your car and ask if you would like Krispy Kremes, but it seems like that bidding war could get expensive quickly - now Krispy Kreme has to compete for retail space along Main street, and they also have to bid against Dunkin Donuts/etc. in the traffic routing business. Further, If Krispy Kreme is routing 25% of all traffic past their store, then the municipality in question either has to:

a) Outlaw this

b) Add additional roads, etc. to the road near Krispy Kreme, thus making this road the new "Main street" anyway. It'd drive up the cost of that real estate where Krispy Kreme is.

All in all, I just don't see it being economically viable for anyone.

Back when in dashboard GPS was a thing, wasn't access to the device cut off while the vehicle was in gear? I wonder how that will interact with an autonomous vehicle.

There are some interesting financial issues where driving me, slowly, past every McDonalds would waste a lot of time and gas. How much gasoline will I have to buy because a retailer purchased X number of drivebys?

For a variety of interesting and irrelevant dietary reasons I don't eat at McDonalds. Presumably a smart enough socially connected car would understand that and not waste my gasoline, time, mileage, and a restauranteurs purchased drivebys so it would not route me past McDonalds. There are two amusements. The first is lying and suddenly according to social media 90% of americans are now following a paleo vegan diet, or never eat any cuisine except Bolivian (because to the best of my knowledge there are no centers of Bolivian cuisine anywhere nearby me) The second is the car equivalent of "swatting" where you fool a social network into believing the victim is a huge supporter of NORML so it routes the car thru some interesting neighborhoods, or endless fun with rightwingers who are now "in the closet" per social media so they get routed past certain bars and bathhouses and such.

One significant problem is autonomous vehicles would remove the need to pay as much attention outside... so advertising would seem fairly pointless. I use mythtv at home to skip legacy TV network commercials; coming up with an exciting new technology to tailor TV commercials to me, which I'm going to skip anyway, sounds very much like horseshoe mfgrs trying to tailor their horseshoes to appeal to the new automobile invention a century or so ago. If no one's looking out the window any more, why spend all that money on signs? Just send them email spam. You can sent a lot of social media spam for the cost of running a giant neon sign 24x7 for a decade.

The modern cars I've driven with LCD dashboards (modern as in like MY2013) have all allowed screwing with everything while driving, even GPS. I remember when it wasn't allowed, but at some point they relaxed those rules?

This brings up very interesting ideas and questions that will happen with autonomous cars.

Routes and retail along those routes will enter a new era. Building a store you have to take into account algorithms of the vehicles. Slight changes to a widely used system could shut down a business faster than individual human choice (although it may also make other businesses seen more as it distributes traffic across more roads - hot spots could become medium and less travelled areas could become more lucrative).

Also, wouldn't it lessen ad hoc pulling into McDonalds on the way home? If the machine is in control then you might not go get a donut even if it is along the route, you'd schedule it in more ahead of time. In the same way e-commerce is more direct, traveling will be more direct and the items at the checkout are not as distracting, same as passing other stores, you'll be busy doing other things like passengers possibly.

I don't see any reason to believe why you are in any less control with a autonomous car. If anything, you would be in more control because making that quick decision turn into McDonalds would be as stressful as pushing a button.

You think there's going to be a "McDonalds" button on the dashboard? No, you'll have to press half a dozen buttons to reroute, and so you'll have past the restaurant and won't bother. Whereas currently, your control is direct and immediate.

Although (to argue against myself), why shouldn't there be a display that shows you the upcoming businesses along your route, that you can click on to reroute? Perhaps businesses who pay extra get slightly larger visuals or video ads, or get shown for a larger radius (for example, the default is to show businesses just to the immediate left and right, but pay extra to get shown a block to the left or right). Quick, file a patent. :)

I use Waze on my phone for my car nav. When I am stopped, it will often pop up a small advert for something near by (usually offering some sort of deal) with a single button that says something like "Go there". Tapping that button will reroute me. If this was built into the car nav of an autonomous vehicle, I don't see why it would need to be more complicated than what we already have in other apps today.

There is limit for decisions for entire day one can have some if some decisions are made for you, you can focus on other things. Technically it is a small slice of your decision allotment but I guess if software can assist me then it is great. However it should not apply to people with less intensive mental labour, for those this might suddenly become habit forming for whatever purpose car decides to stop over or not.

So in general it is ok for a car to stop by only if the car can roughly estimate your health, blood sugar level etc. I mean if the car can assist me for better living that would be great!

my 2c

Well, one solution to this is to have it show you the route in advance and have you confirm it (or even edit it first). If it picks an obviously suboptimal route, you know something's up. If Krispy Kreme is already on your optimal route, then too bad.

Basically, as long as the thing doing the navigating is also showing your the route it plans to take, then it can't pull any funny business like this without you noticing.

This shows surprising parallels to Net Neutrality, highlighting the importance of getting this legislation correct for future technologies.

We almost need a new "Bill of Rights" for the 21st century to deal with all the ambiguities brought about by technology.

What if your news article keeps buzz marketing Krispy Kreme?

Tell it to stop.

Reply on news.ycombinator.com