Hacker News Reader

Gill Pratt is not "some dude" -- he's a robotics researcher with a PhD from MIT, and was a DARPA program manager [0].

[0] http://corporatenews.pressroom.toyota.com/article_display.cf...

To engage with the substance of his argument a bit more, his argument is that perfection is a long way away:

> “Historically human beings have shown zero tolerance for injury or death caused by flaws in a machine,” Pratt said. “As wonderful as AI is, AI systems are inevitably flawed… We’re not even close to Level 5. It’ll take many years and many more miles, in simulated and real world testing, to achieve the perfection required for level 5 autonomy.”

I can believe that, but I also disagree that we need to be there for it to be useful. Level 4 is enough for large scale deployments.

Lots of people make this argument. I have a counter-argument: when a technology is convenient or appealing enough, people will overlook even very obvious safety/health problems. Consider texting while driving, drinking, smoking/vaping, even driving itself. Cognitively, we know that there are a bunch of things we do that are dangerous, but we do them anyway out of habit or convenience. The argument that assumes we demand "zero tolerance" seems to assume that every person is a run-5k-a-day, celery eating, take-the-bus, machine. That's far from the case. We do tons of things knowing they're bad and risky already.

I think it's far more likely that there would be a three step process for this technology to be adopted: first it's early adopters and everyone else is like "whoa that's too far" (I think this has already happened), then people start to uneasily use it, and when it's good enough they realize "oh holy crap, the machine can drive while I watch Netflix! this is awesome" and then they'll use it constantly. Soon, it just blends into the background. Maybe it isn't 100% safe, but what is?

I don't think you'll get the truth out of focus groups, either. Try to hold a focus group on self driving car adoption and 99% of the people will tell you "only when it's perfect." Probably if you ask the same focus group if they text/eat/put on makeup and drive, 99% of them would tell you "absolutely not" when I'm sure that 100% of them do. I think that once a somewhat good enough assist gets into peoples hands, it won't stop.

For a real-life example, just look up "Tesla autopilot almost crash" on YouTube. There are plenty of people who let Autopilot drive when they really shouldn't.

With software, it's always easy to make something appear to work. All the corner cases make something take 10-100x longer than the functional prototype.

This became painfully obvious with the article floating around where a car misread a stop sign with a small amount of marking as a 45MPH speed limit sign. It's pretty clear that if a car is mistaking a red sign for a white sign, we have a very long time to go before the car can safely drive itself without a human observer to intervene when needed.

When it comes to automated driving, we need to keep our optimism in check and know that it can take decades before we have safe robotic taxis. They may become legal in countries with lower bars for safety before they are legal in the US.

The big problem, I think, will be when the drunks get into accidents because they will turn on autonomous driving, and don't get pulled over. Instead, they won't be sober enough to observe and intervene when the car screws up.

> a car misread a stop sign with a small amount of marking as a 45MPH speed limit sign

That's what happens when the sign were made for humans. They should start putting QR codes under traffic signs, with the human sign as a fall back or as a second point of reference.

How can you verify at a glance that the QR Code associated with a human-readable sign is correct?

Not that I think QR codes are a good idea, but why does this property matter?

People can be dicks to each other in myriad ways, I don't find these arguments that you could trick self-driving cars super compelling.

I said this in my comment: "...with the human sign as a fall back or as a second point of reference."

You wouldn't remove human sign identification from the algorithm, you just supplement it with the small QR code under the sign.

QR codes are a goddamn eyesore. I'm voting against any city counselor who thinks they're a good idea. Find another way.

"This became painfully obvious with the article floating around where a car misread a stop sign with a small amount of marking as a 45MPH speed limit sign. It's pretty clear that if a car is mistaking a red sign for a white sign, we have a very long time to go before the car can safely drive itself without a human observer to intervene when needed."

I think you misread that article.

Someone built their own toy implementation of an CV sign recognizer. Then, with full access to the code, they reverse engineered visual distortions that could seem innocuous to humans but trip up their model.

It's not a real world issue as far as I'm aware (though obviously signs can be misread or missed, but not invisibly vandalized)

While they did use their own simple DNN classifier (because they couldn't find one), it was treated as a black box for purposes of the attack. A more robust classifier (probably using multiple methods) would be more resistant to attack, but the attack would still work, just the changes would be more obvious. At some point you hit "human-recognizable" but it's unclear where that point is.

As long as you have to intentionally modify the sign to cause confusion, as opposed to doing random vandalism, I don't think this is a problem at all. It's like a person spray painting extra loops to change a 30 into an 80 or sticking a home-made sign over a real one so it's indistinguishable to most drivers. People aren't going to do that any more than they already do and it'll still be just as illegal.

Sure, some image classifiers can be fooled by some vandalism that requires a high degree of technical sophistication, but

a) Humans can be fooled by vandalising a stop sign too. The difference here is that the change is imperceptible to humans... but does this matter?

b) A lot of self-driving work leans heavily on maps which will know to expect a stop sign.

It does matter if half the cars read a speed limit differently than the other half.

The other day, I came within two feet of a crash because the other driver completely disregarded a red light.

It's pretty clear that if a human can't even pay attention to a light directly pointed at them, we have a very long time to go before humans can safely drive cars without a computer to intervene when needed.

One thing worth mentioning (even though I agree with the gist of your point) is that machines make the same mistakes consistently. This is pretty rare for a human. Imagine 10% of the cars on the road are autonomous and they all make this same mistake.

There are a lot of road design features that come from the fact that humans do make certain mistakes consistently. For example, the US's love affair with four-way stops comes from drivers' inability to safely navigate uncontrolled intersections, even when there's no particular reason they couldn't. Stoplights usually have a delay between the red light for one direction and the green light for the next direction because driver's often run fresh reds, or get stuck in the intersection and need to clear out. There are tons of warning signs because drivers can't be trusted to notice something as simple as a sharp curve coming up.

Of course, the road system is designed for humans' foibles, not computers' foibles, and computers will have to deal with that. But the bar is low.

Computers aren't really as consistent as you say, either. Obviously, a deterministic machine will produce the same outputs for the same inputs. But when your inputs are camera data from the real world, you'll never get the same input twice. For example, my car sometimes misreads or fails to read speed limit signs, but it'll usually read the exact same sign perfectly fine the next time I go past.

What models of car read speed limit signs?

Teslas can (although the second revision of the Autopilot hardware doesn't do it yet).

The rental Astra we had the other month did.

I wonder if in the future the law will say self driving cars can ignore all those signs when the car chooses to. Obviously there will be a lot of legalese about it, but most signs can be safely ignored already if you understand all the hazards - this is why people tend to just slow down at stop signs not come to a complete stop.

This is better for the environment, there is a lot of energy lost in a full stop for a stop sign that could be saved: less air pollution/CO2 to deal with.

I think it's the opposite. Humans constantly make the same mistake like being drunk, tired, using a phone. With autonomous cars every single collision will be investigated - where the software is at fault a new improved version will be pushed to all cars, elimination the problem. This systematic improvement will mirror the improvement of airplane systems, but with a much faster time frame

It's important to point out that the WayMo project in Phoenix is only barely getting started, only serves limited areas, at limited times, and presumably in limited weather conditions; and most importantly that there's still a human driver in the vehicle at all times. A full-scale metro-wide service would need to operate at all hours, in most any weather, and without a human driver. That's not happening within four years.

>WayMo project in Phoenix is only barely getting started, only serves limited areas,

Sure but is this because of capability or number of vehicles?

>at limited times,

Sure, because drivers don't want to work late at night.

>and presumably in limited weather conditions;

Why's that?

>and most importantly that there's still a human driver in the vehicle at all times.

Sure, but this isn't due to capability, but for regulatory reasons. As far as I know, the firefly vehicles (the cute little bubble ones) don't have steering wheels, and were in use 3 years ago. This argument doesn't make sense to me.

>>and presumably in limited weather conditions; >Why's that?

Because Phoenix has extremely low precipitation: http://www.usclimatedata.com/climate/phoenix/arizona/united-...

>some dude from Toyota, who don't seem to be very invested in this space, tell me he doesn't think it's going to work isn't very convincing

Similarly, opinions from those heavily invested aren't convincing.

Are there people with credible experience, but without bias either way, that are giving projections on when it might be viable?

John Leonard at MIT is one long-time (going back to 2007 DARPA challenge) researcher who is very skeptical.

e.g. http://searchcio.techtarget.com/blog/TotalCIO/Driverless-car...

"I think that driving exposes fundamental issues in intelligence, fundamental issues in how the brain works. And we might be a very long way away.”

I think his research area (SLAM) might blind him a bit to the engineering side of things. Simultaneous Localization and Mapping is great if it works, but for something with high enough value... you can just map it from the start, and it makes your problem much easier. Which seems to be the approach people are taking.

Mapping does make things easier and it arguably jump-started the current wave of autonomous vehicle work. But then you have to move beyond a precise map of how the world is supposed to be to how the world actually is including all the unpredictable behaviors of humans in the space.

Having listened to Leonard speak, his skepticism seems to mostly be around how you deal with all the things like left hand turns in busy traffic, police waving people around an accident, etc. that happen on a daily basis.

I think that for a large enough benefit we will adapt how we interact with these systems.

Before automobiles it was normal for people to walk in the middle of the street, but we adapted (some would argue this was a bad thing ofc).

I don't think it's a prerequisite for autonomous systems to be the same as humans, the benefit is just too large for us to wait.

MTA has 100+ years of legacy track. Imagine trying to retrofit anything into a 100-year-old codebase. Ouch.

In effect, aviation and automobile infrastructures likewise have 100 year old code bases. And for cars, it varies by state (and county and city but mostly by state).

It's not uncommon to slap a DRO kit on a 100yo machine. 100yo bridges routinely get upgrades in surface coating and/or instrumentation/electronics.