The Next Leap in Self-Driving: Prediction
olivercameron.substack.com
olivercameron.substack.com
They're very jerky at intersections where there's even a mild amount of uncertainty and really only go when it's very clear. It reminds me of lots of first time drivers reacting to new situations where safe is the best way forward. But if we had pure first time drivers, we'd be a gridlock at even a mild amount of traffic that enters and leaves the roadway.
Looking forward to this next set of 5 years. Hopefully us drivers will get replaced in 10? maybe?
W.r.t."self driving winter", I think Waymo are aiming to protecting themselves from that situation by developing close relationships with regulators, highlighting the "we're not them"-part and sticking to their safety-first approach.
Again, the measuring stick is real drivers. And real drivers are really bad at this stuff too.
For the comparison to hold you need to do the following:
- discount all the hours when humans are in adverse conditions and autopilot is not normally engaged
- remove motorcycles and other vehicles from the pool
After you do that Tesla is still much less safe than your average driver. Their whole spiel of taking the easy 80% in miles of driving and claiming victory has been debunked so often now it is getting boring. It's the remaining 20% where the fatalities are and which - coincidentally - are the hard bits.
The only story I see is politics and complexity, as opposed to moral clarity.
People (and the media) are going to focus on the accidents that would not have happened if self driving did not exist.
But machines make different mistakes, mistakes that humans would never do, and that's psychologically not acceptable to most people. All the actual accidents will be "freak" accidents that a human could never cause, and that scares people a lot more than "human" accidents like a drunk driver running someone over at a pedestrian crossing.
If I get killed by a drunk driver, my wife will at least have someone to rage at.
If I get killed because my car randomly decides to drive itself into a high up truck and I basically died because of stupid technology. I would hope that it doesn't end up killing my wife too out of rage that she's going up against a major corporation now.
That's kind of maddening.
It psychologically like being a passenger in a car with a good driver who has a history of epileptic seizures.
I think what would be accepted is if the self-driving car would have capabilities that surpass the best human driver (as opposed to the "average" driver). But we're a long way from that.
I agree with this but I think it implies that people also will not accept safe self-driving cars that take forever to get anywhere in traffic. Especially since a decline in average traffic throughput makes things worse for everyone in a car, self-driving or not.
I think this makes the problem much harder, since a lot of people do demonstrably take risks in their driving now, in the interest of saving time (even knowing intellectually that lives are on the line).
If you assume people drive 10K per year at an average speed of 33 mph, you’re driving ~300 hours per year. Slow that by 25% and it’s an extra 100 hours per year. If you assume 7.5 hours of sleep, you’d spend an extra 1.6% of your waking life in a car because of the slowdown. That’s killing a lot of time, even if not directly killing people.
frankly I don't think any of the teams who have demos are as close as they hint they are. there are just so many exceptions that it becomes maddening.
look, people road rage over the driving habits of others and the computer while it won't road rage has to react to those who infuriate others with their driving either because its reckless or just bad driving
some of my favorite conditions....
people stopping to let you out, how does the car know? people going out of order at an intersection?
* I do not let it drive where I think the situations are complex and obviously it does not have the ability to respond to traffic signals or make turns at intersections.
Yurikamome trams in Tokyo runs in L4-L5 and Tsukuba Express runs in L2, in SDC levels term. There was also a L4 PoC in Yamanote line so that’ll probably come by the end of this decade. Elevators had been L4 for like half a century as well, by the way.
In other words, getting trains to L4 is straightforward, and there's been a rollout for half a century. Making the jump to "doesn't even need the driver to handle emergencies," that's a task comparable in scope to getting cars from L2 to L4.
Would be really interesting if Waymo could prove this out somehow.
I'm driving southbound on Fair Oaks in Sunnyvale, approaching Arques. On the corner is a mini strip mall with a small parking lot. Out of the corner of my eye, I see a young child, maybe 4 years old, running around that parking lot being chased by his mother or caretaker.
Now, being well off the road and out of its universe of conditions that would be considered by an AI, he would be ignored. But I just knew, somehow, that this was trouble.
The kid, thinking this is a fun, impromptu chase game, then sprints directly into traffic right in front of me. I stop with a foot to spare.
Edge conditions: it's what's for dinner. (tm)
A human could see a group of kids looking in the direction of the street at someone occluded from view and realize the possiblity of a child dashing out onto the street well in advance. The self-driving car would have to detect, classify and react after the child comes into view. It is this kind of common sense AI that we cannot engineer our way out.
A couple days ago there was a viral post about how self-driving cars react to a 35 mph sign vandalized to look like 85 mph. A human would know that the sign is wrong - the self driving car would need this 'use case' programmed in - maybe have location based speed limits programmed in.
Reading the article:
> ‘It is worth noting that this is seemingly only possible on the first implementation of TACC when the driver double taps the lever, engaging TACC.’
It isn't a strict self driving situation, but makes for great click bait. I can't imagine that this isn't an easy fix (Tesla's are also easy to update collectively via software update, try doing that to human beings).
We should also be very clear that 88% success rate given for object detection and identification is against a relatively fixed set of static images[1]. And as far as I can tell from the paper[2], there's no indication of how long it takes to come to a conclusion on any single image, and no description of how much hardware was used to reach this rate of success, which to be honest, I'm surprised is so low.
Given all of that, it does not seem reasonable to assume that this 88% success rate has anything to do with how well object detection works in real-time using on-board computing power while the car is in motion. So, don't start planning to throw out your driver's license anytime soon.
[0] https://paperswithcode.com/sota/image-classification-on-imag... [1] https://en.wikipedia.org/wiki/ImageNet [2] https://paperswithcode.com/paper/self-training-with-noisy-st...
The best strategy for a resolution isn't to be accommodating. It's to physically shift your entire body to one side and make no eye contact to make it absolutely clear to the other person which side of them you'll be walking on.
And don't even get me started on the "automatic" windshield wipers.
Slowing down is also the best option for a tire tread or a shopping bag. The bag being less safe than you imply if it is filled with a box of nails or happens to wrap around a driveshaft or exhaust manifold.
Too many people my age and older will be apprehensive about giving up control, despite it likely being safer. So I think it is fair to say that in 20 years there will be many people who learned to "drive" with a self driving car and their parents and grandparents will refuse to give up driving themselves.
Once proven safe (or if a manufacturer directly insures the product) the cost to adequately insure a vehicle (as prices of what you hit continue to increase), especially an older vehicle, will increase. With income inequality where it is (and state minimums lagging) it has already become common to have underinsured drivers (and the upper middle class are already paying more through uninsured motorist protection).
When it comes to police enforcement, but especially jurisdictions that use traffic stops to find or create crime to fundraise; the expense of having traffic police in a world where > 50% of cars can do no wrong flips the model.
Finally, there is a large group of people who will not drive a car that's 7 years old or has over 100k miles; combining with less trained ability to do maintenance on more computerized cars. These people claim to be in control, but at one check engine light away from doing whatever a dealer tells them.
There's nothing wrong with driving a newer car and relying on the dealer service. Sure it's expensive, but if you can afford it then so what.
Maintenance needs in any modern car mostly comes back to value engineering. It's literally someone's job to optimize "how cheaply can we make this part, that it breaks just often enough that people won't complain".
The exact same thing is already happening for electric cars, and you'll need to spend exactly the same amount of time and money on service.
Furthermore, almost none of the service on a modern car is related to the combustion engine itself. It's mainly electronics, linkages, wheel beqrings, shocks, suspension, rust damage, tires etc.
If you think about it, a car that's done 200 000 miles at EOL has only been running for around 200 days continuously. A generator, or the engine on a ship, train, airplane etc. runs more than a full order of magnitude longer.
In my case, the “myth” very much matches my experience.
Seems obvious to me that the people who are economically forced by insurance rates into self-driving cars first would be the worst drivers. That means arithmetically that the remaining drivers will be better than average. So the rates for everybody should go down.
Or, another example, a big parking lot with a few buses, people unloading their stuff from the buses (let's at the end of a long trip), relatives/parents/friends pick them up at the parking lot. Relatively dense, many people moving, but still cars come and go. Plus a lot of packs/luggage here and there. Usually in these cases more cars want to enter the parking lot to pick up others, but of course people want to continue unloading, so some kind of balance forms.
What happens if the user is drunk? Well, naturally, the cop tells the car that it's a cop and just pick a different route.
And we shall see how well it will work.
Probably it'll take a (few?) decade(s) before a self-driving car should think it can safely wade through a crowd instead of going back and finding a good place to wait for user input :)
They’ve been peddling that these Level 5 Waymo cars were just around the corner. But they refused to disclose how the cars actually worked, and kept saying that their black box is so mysterious, that their scientists themselves couldn’t understand it, so there was no point in having the rest of us, mere mortals, try to understand it. Lies.
We are a long ways away from being able to trust a robotic car with our lives.
But to their credit, at least they realize that requiring a human to take over vehicle controls in a microsecond, is unrealistic. Ahem! Looking at another car company that thinks this is acceptable.
Objectively though they have started performing rides with no drivers in AZ.
https://www.theverge.com/2019/12/9/21000085/waymo-fully-driv...
A simple example is to be in a lane next to a lane that ends. Everyone sees the same signs that indicate that lane is ending; everyone knows the cars in that lane are going to get over. But there are a variety of strategies for doing so: get over early; wait until the last minute; accelerate to merge; slow down to merge; use the opportunity to pass; etc. Some drivers are nervous and cautious; some are aggressive. The way the cars move on the road, the signals they provide like brake lights and turn signals, are a form of communication that you, the potentially yielding driver, can interpret and make decisions about how to deal with the merge. Or prevent it! If that is your inclination.
This seems like a hard problem IMO. Perceiving objects is essentially about physics--detecting and recognizing. Next step harder, making physical predictions e.g. how far a moving object will travel in 200 milliseconds. Maybe that's enough to keep a self-driving car safe, but it won't make it very efficient in heavy traffic (which is the norm in urban areas). To interpret intent as part of a prediction seems way harder than even that. Reminder that when humans start driving they typically have 16-18 years of continuous social learning... and even then it usually takes years to become a comfortably safe driver in traffic.
Of course this would be way easier if all the cars were self-driving, all at once, but that seems like a really hard way to implement this technology. I think you'd have to have a city with very strong leadership that defines a small area that is self-driving only (working with one or more private companies to provide the vehicles), and then slowly grow the boundaries of that zone.
https://en.wikipedia.org/wiki/Flocking_(behavior)#Flocking_r...
So it might be a relatively easy problem for cars, if all the cars could be made to follow a common set of rules. It gets more difficult with humans who may follow different rules, or even actively exploit the rules of the automated vehicles.
Yes, one way to make a problem easier to solve is to make the problem easier. (But how...)
I admit that if people acted exactly like birds, self-driving cars would be easier to implement.
* Object detection is not solved.
* Even if it were, prediction requires a theory of mind. What does the other person see, know, and want to do?
* Even if we solved prediction, driving requires social interaction, coordination, and cooperation.
I think self-driving car companies will try solve this with remote internet-connected operators that step in and remotely flash some lights or something to communicate. Of course these people will be overworked and underpaid, will have no skin in the game and not really care (at the end of another long shift) about your particular interaction with their company's vehicles...
Sure, a fallback remote pilot is probably going to happen too, but that's a very big can of worms. Just think about what happens if the remote pilot wants to take the car through a rail crossing and the connection drops. Whoops.
So some kind of consistency would help. If the self-driving car decides to let the other car in front of it, then slow down, wait a bit, then okay, chance missed, go ahead.
How else would it work? This part is classic reinforcement learning. I don't think this policy part is the hard part. (Eg. similar speed merging, and from-stop merging, or stop and let other car in.)
Probably each brand will have a personality. (Based on their timeouts and other parameters.) And humans will adapt quickly. (And naturally the self-driving cars will change over time too.)
Chat requires an amazing amount of implied context. Merging? Not so much. Driving in general? Yes, that does, because there are strange edge cases. (Like what to do when you are lost, or the car breaks down, etc.)
I would think that perception in poor weather remains a real challenge, but certainly we’re leaps and bounds ahead of where we were even 5 years ago.
Now what bothers me is that leaders in the industry say without batting an eyelid that perception is solved, given that they witness this type of events on a regular basis.
Tesla has made a name for over promising, under delivering on fsd, and refuses to consider lidar as a member of their sensor suite.
I don't think you can adequately compare two companies with vastly different approaches.
You can see it on reddit here: https://old.reddit.com/r/teslamotors/comments/f4fbfu/nav_on_...
In the video (https://reddit.com/link/f4fbfu/video/5yg4oclye5h41/player), skip to 0:38. There is a construction worker walking at a constant speed towards the car's lane. The car could only assume he would keep walking and swerved and stopped.
For a human, we'd probably assume the worker is paying attention and will not walk in front of us. Though at times that also ends up being wrong.
Back on topic, one thing that is invaluable for human drivers is eye contact. I assume that should be taken into account here, ie. if the pedestrian makes eye contact with the vehicle we can assume he has seen it. Otherwise it will result in a lot of unnecessary swerves and stops especially in towns where pedestrians walk right up the side of the carriageway.
It's pretty common for motorcyclists or bicyclists to make eye contact with a car driver who proceeds to pull out in front of them anyway.
This is probably in part from car driver brains not interpreting bikes as physical threats. It doesn't have to be deliberate. It can also be the driver scanning for "car" and not seeing "car".
A self driving car is on the other side of that size equation, but eye contact might also be assumed by a pedestrian to mean that you will stop. Especially if a "driver" makes eye contact who isn't driving.
In the end it's best to slow down and not assume. Humans often make dangerous assumptions, for example going around blind turns or over crests too fast to avoid something that might be there. Most of the time it's ok but eventually somebody gets killed. The rational thing to do is make the software safer than humans, not emulate them.
So what do I do? Get to the corner and stubbornly refuse to look up at any of the cars, and don’t really face my body toward any possible crossing. Not being able to wave me on (I’ve always found being waved in front of a motor vehicle somewhat menacing, tbh), the drivers quickly clear the four-way stop and I can cross without worrying about causing a jam.
Anyways, the AI will have a lot to learn of the crazy habits people build to keep from being lawfully murdered because “but cars”.
I know they're trying to be nice, but it ends up slowing everyone down: suppose a car and a bike are approaching a four-way-stop intersection with the car clearly going to arrive a couple seconds before the bike. As the cyclist, I would assume the car will come to a stop or almost-stop, wait about a half or one second, then start moving. With that assumption in mind, I'll adjust my speed so that I slow down for the stop sign (prepared to stop in an emergency) and roll through right behind the predicted position of the car, without losing too much momentum.
If the car takes their right of way, this is great. But suppose they decide to stop completely and wait for the bike to go through first (against the right of way). Now as the cyclist, I also have to stop completely, since I don't know when they're going to start moving again. If we make eye contact, I'd feel safe to go, but I feel like that's reinforcing bad behavior. Instead I refuse to make eye contact and force them to go first, as they should have initially.
I.e., create maximum confusion as to your intentions? Doesn't sound great to me.
But long before that, you'd need to realize a camera suite with an effective resolution between 200 and 500 megapixels, sampling at 60 Hz. We're nowhere close to that today.
He is not privy to what his competitors have been working on, in any case, except whatever has been announced publicly. (Unless he is doing industrial espionage.) Maybe it's just that Voyage's cars weren't predicting up until now?
Heard of control systems waiting for seconds before acting on new information? So don't assume flawless, perfect technology.
If my car is driving, instead of trying to predict the behavior of other objects I want it to do something like a minimax search and do something that will be safe in the presence of the unexpected, not the expected. Prediction sounds like the complete opposite of good driving practices like the rule of following at least 2 seconds behind the car in front of you, which is specifically there to prevent accidents in the rare case where the car in front of you on the freeway does NOT behave in the predicted way of continuing to go 70 mph in the middle of the lane.
This is called defensive driving, and you're wrong about it not being based on prediction. Defensive driving predicts that every wrong thing that can happen will happen and positions the vehicle to minimize the harm when they do.
Fundamentally, if you don't predict that the car up ahead may swerve into your Lane and slam on its brakes, then you're less defended when it does. Defensively choosing what to do and where to be at any moment is based on predicting it happening and acting accordingly.
For example, if you are driving behind a car on the highway, and the car _in front_ of that car puts their brake lights on, you may want to pre-emptively start braking now now. That's a type of pro-safety prediction made based on your understanding of the likely future that driving AIs should also have.
A self-driving car that takes forever to get anywhere could easily fail in the marketplace, even if it is clearly safer statistically.
The thing about very safe driving strategies is that humans can already choose them at any time, without buying anything. I, too, can sit and wait for a roundabout to fully clear before going.
The fact is that a lot of people do a sort of risk-cost-benefit analysis in addition to safety analysis when driving. I'm not sure a car optimized heavily for safety, at the cost of efficiency of travel, will meet expectations.
https://66.media.tumblr.com/fdd6a0be1711940bd5a9e0f31e9f01ac...
then we can have automation without job loss into the bargain.
Hard to sell indeed.
Should the downstream system trust the error bars with certainty?
It's hard to define what a right prediction is. The correct prediction is whatever makes the car safe and effective. If an action leads to a situation with a low probability but very catastrophic outcome, the system should not take that action, even if the prediction correct with its error bars.
Basically, autonomous vehicles almost entirely assume the worst outcome in all situations, and waits until it is safe. So at a 4-way stop, it will wait for all cars to be out of the picture before going, which results in horrendous performance if the intersection is busy.
Reason for this is too many times I've just narrowly avoided a collision because someone forgot to cancel their signal, or they were signalling ridiculously early for a turn they were going to make further up, or even that they just activated the signal accidentally.
Not to suggest prediction is bad, but just that also not treating turn signals with too much authority is safer.
On my motorbikes, making good progress depends on good predictions, and that means predicting that people don't see me, that they make rash and impatient decisions, that tourists on rental bicycles will do stupid things without looking, etc.
That's some massive handwaving.
Tesla had a lot of presentations about the complexities of detecting the right light to use in a crossing with about 20-30 traffic lights for example.
Depth detection is also crucial, and as you wrote there are lots of strange objects/animals/people to handle as edge cases.
I think the next step after proper visual prediction is interpreting the noise around. For example, a human can hear the ambulance siren approaching and slow down in a junction without seeing the vehicle. I think sound interpretation is very important especially in city driving.
Staying alert on watch is something that nearly every solider, security guard, sailor, will have to do. Where you basically stare into the dark in case someone comes.
There are a number of ways to mitigate the inevitable failure of attention:
* Having a parter;
* Doing short shifts (change with partner);
* Very simple non-distracting tasks - like reading out to your parter a figure at a glance.
You certainly should not:
* Be on your own;
* Have a mobile phone on you (all screens should be locked away while moving).
It's so fucking arrogant that Uber didn't care to implement the most obvious risk mitigation. The cost of an additional person on watch is negligible compared to the cost of their self driving care program. My gut feeling is Uber simply misled regulators with how developed their self driving car program is, so it could continue to mislead investors.
If they didn't willingly disable safety features the crash wouldn't have happened.
It wouldn't have happened with the shittiest auto emergency braking system that relies on 0 Ai and 0 predictions.
Taken to the extreme this causes problems, but gently slowing down in the case of extreme tailgating is a response to slow reaction times not direct path prediction.
Driving is cooperative not adversarial, even certain classes of illegal behavior are predictable.
But software in their current state? Is it possible to provide enough logic to handle all cases like humans? Can you do that with pattern matching, feature detection, decision trees, bayesian models and anomaly detection alone? And if we do, didn't we just replicate a human driver with all his flaws?
Look up the work of David Cox[1], Karl Friston et al. A fascinating variant is Contrastive Predictive Coding[2].
[1] https://www.youtube.com/watch?v=P0yVuoATjzs
[2] https://ankeshanand.com/blog/2020/01/26/contrative-self-supe...
The state of the art in machine perception is still far from human level perception, regardless of how quickly progress has been made. Prediction is a function of perception, so it's also bottlenecked on perception getting better. Let's not confuse recruitment-oriented blogposts with credible scientific writing.
Essentially giving them 'The Flash' like vision. You could then have all kinds of predictive models based on various patterns.
Check out www.sebenz.ai