Yes. That would be the AV permit that the cars operate under. Also as they are programmed to obey all road laws, it's exceedingly unlikely they will cause any moving infractions.
> Who's responsible? Who gets the ticket?
Cruise. Their name is on the AV permit, it's their insurance.
> spot essentially identical to where it was.
Before it was in a lane of traffic next to a parklet, after it was pulled over partially into what looks to be a parking spot. The SFPD officer also was no longer next to the vehicle.
> how these cars are allowed on the road
At the end of the day the argument is that these cars are safer than the average driver. Having spent a lot of time in SF, I've seen a lot of these cruise vehicles moving about. I remember watching them in the early days in the Dogpatch, when an errant cone would end up with them being stuck until manual action was taken. Slowly they got better and smarter.
Yes, it's a hard transitional time. Mistakes may happen, but the same is true for humans. And not all AV technology is the same. Cruise has limited itself a great deal so it doesn't try to take on the entire world at the same time. Just the streets of SF, which are hard enough.
How many Cruise traffic accidents have you read about? How many people injured or killed?
Arguing the probabilities of an outcome is an engineering-centric approach to selling these cars to the public, and it doesn't land well IMO/in this thread from others.
It makes a false equivalence b/t how we value human agency's role in outcomes, and something other-than-human. Most people seem to prefer the risk outcomes of something I decided, vs. a faceless control system.
> At the end of the day the argument is that these cars are safer than the average driver.
The last data I saw said otherwise, and that data was primarily collected in ideal conditions. That has been a few years, but these companies are not forthcoming with their data.
I think this is mandatory in most states and countries, at least for civil liability up to some limit. You can purchase more, of course (and I recommend to do so, given that the minimal liabilities in each state haven't really kept pace with needs).
Search a jobs aggregator for "Taxi Driver"?
Thought experiment: imagine a particular self-driving algo were to do something that would warrant a human driver getting their driving licence suspended, would that mean all the cars with that same algo would be simultanously suspended, or just that one car?
1. Easy to implement
2. Doesn't punish fleet scale (compare: suspending the whole fleet means a fleet that's twice as large would be suspended twice as often)
3. Punishes the fleet owner in proportion to the frequency of the error
4. Is exactly correct when the cause is really a hardware problem specific to the car
If the software in a commercial aircraft were to be found defective, would we really be OK if only the individual aircraft in which the fault manifested itself gets grounded (assuming that aircraft isn't itself already on the ground in thousands of pieces spread all over a hillside...)
Surely the FAA (or international equivalents) would issue an airworthiness directive which would apply to every single example of that specific type of aircraft?
If it's non-safety-critical, then it needs to be fixed asap but nothing gets grounded.
If it's safety critical, then yes, all planes are grounded.
But, I'd argue FAA levels of safety aren't suitable for cars. If a car has an engine failure or even a structural failure, they can usually just stop. Planes don't have that option.
I had supposed we were thinking primarily about software faults and specifically those relating to the self-driving bit of a self-driving car.
Best case: self-driving car gets lost and is found with flat battery miles from "home" (?)
Worst-case: self-driving car kills cyclist/pedestrian and doesn't even notice or stop.
We should spend some thought on how to handle the latter because it will happen sooner or later. To be blunt: if the algo kills, why wouldn't you ground the algo (all instances of it)?
It depends what the alternative is. If it crashes less than humans, then you keep running it despite the flaws.
Well 40,000 people in the U.S. die in car crashes every year currently, why do we let people drive?
(Of course, you don't want to fork too much, because there's probably a certification per algorithm needed. But it might make sense for Google to officially field a thousand 'different' algorithms, so that a single incident doesn't ground the entire fleet?)
Either one of two things will be the case with this approach:
1. The algorithm will be too imperceptibly different to be treated as a different program. Adding some tiny decimals somewhere won't make it unlikely to make the same mistakes, and the entire class of algorithms will be treated as one.
2. The algorithm will be different enough to make different decisions, which means one program is probably "better" than the other(s) and they'll be taking a bigger risk by deploying cars to the road that are more dangerous than others.
Your second point would lead to a ban on all but one self-driving car systems, when taken seriously?
Just have variations that don't dominate each other. Variant A could be better at some things, and variant B could be better at other things, etc.
(Otherwise, we should use your argument to only allow exactly one company with the 'best' system on the road. Instead of allowing competition.)
Google could even officially outsource the different variants to different nominally independent companies, if that's what's legally required to not get the whole fleet grounded.
* It might be a software issue (maybe the cars self-destruct when they encounter a solar eclipse -- silly example, but if the cars malfunction on events which are rare but wide-spread and correlated this could cause an emergency).
* If it is a hardware issue, it must be understood how a hardware failure can put the device in a dangerous, still semi-functional state. If one car was able to get itself pulled over for driving dangerously, then there'd be an unknown number of cars driving themselves dangerously.
* It is good that the fleet is punished disproportionately to the frequency of errors. We shouldn't accept a linear disincentive for putting dangerous vehicles on the road.
We already do..
Putting driverless cars to a much higher standard, makes perfect the enemy of the better. Human drivers are not all that great.
I think you're letting perfect be the enemy of good. In an ideal world we'd stop everything for debugging. Realistically all we need to have a net benefit right now is for the cars to be safer drivers than human drivers on average. As long as the issue isn't suspected of being intentionally triggered or correlated with some sort of large scale event, the world is still paying less of a cost than now even when not grounding the fleet.
A reasonable compromise would be to require autonomous vehicles moving in especially risky areas like highways to have a human inside with at least a "stop or slow down safely" override. In places where the vehicle would be moving too slowly to cause serious injury we can allow them to be human-less. By defining clear and objective criteria, the decision to ground a fleet becomes much easier to make (particularly for the machine, since you don't need advanced AI to check GPS against a database and apply appropriate speed limiters). If the incident was a serious malfunction of a basic function like ignoring the criteria for human-less operation, the fleet should be grounded. In other cases it wouldn't justify preemptively grounding the entire fleet (but may justify it after the severity of the issue has been assessed, probably by a group independent of the company responsible for the vehicles).
But more practically: what could a self-driving car do to get it's licence suspended? Afaik That's a pretty rare punishment outside of two cases: drunk driving or street racing.
Should that investigation reveal a reproducible behaviour, a recall or general suspension of vehicles running that particular software version would seem like a reasonable response. Otherwise suspending just that one car could be an option.
It's uncharted territory both in terms of the legal system and w.r.t accountability so I guess no one has an answer to that yet.
You could drive the same car down the same road at the same time tomorrow, and it may behave differently. It makes bugs / edges cases impossible to reproduce and fix.
With a human you can just say "why did you do that? Don't do that." Little fine tuned tweaks to neural nets is not so easy.
It's non-deterministic in a practical sense; you cannot reproduce the exact circumstances that caused a certain behaviour. Maybe a bug flew by one of the cameras at a certain moment, or a pedestrian with certain type of clothing passed by. You get the idea.
The camera itself might not be deterministic enough to reproduce the same data with the same beetle, but replaying the log should work.
(Keep the log in a 'black box' on the car, and perhaps only keep the last x hours of data, if necessary.)
I think Waymo already does something like these kinds of replays.
It's not if it's hardware failure.
It's a cost-benefit trade-off. We don't ground the whole fleet of human drivers, when one of them crashes their car, because the benefit doesn't outweigh the costs. It's not a given that the answer will change that much for self-driving cars. Especially for extremely rare failure modes.
We exist on a spectrum of competence. If a type of car has a widespread issue that unexpectedly gets it into a state where a minimally competent non-impaired person can't operate it safely, that type of car should be pulled from the road. However, because humans are so sloppy (0 human drivers should be the ultimate goal), it is generally assumed that most accidents are caused by us, and so we require a significant accumulation of evidence to take a type of car off the road. Without a human in the loop, what's the unreliable part to point the finger at?
(Or perhaps to have separate versions on file, and just load that different version when the first version is grounded.)
> Without a human in the loop, what's the unreliable part to point the finger at?
It's a cost benefit analysis. If the problem is sufficiently rare, it might make more sense to leave the cars running while you investigate. Even human lives aren't infinitely valuable.
Typical traffic engineering in the US assumes a cost of human life of about 6 million USD or so.
Details depend on location / jurisdiction. Some Googling brought up eg https://www.dmv.ca.gov/portal/vehicle-industry-services/auto...
Not in the US, but that's how it would work here for illegal parking or if the driver fled from the cops and the car is never found for instance. Even if the owner doesn't own a driving license, that's just where the buck stops. The owner if of course free to try to shove it somewhere else, but that's on them to make it work.
In other countries with similar laws(Poland) it has been argued in front of court that it cannot be the owner's responsibility to know who is driving their car at any given point especially if there are multiple people who do that usually - it should be the police's job to establish who they are fining.
Then the driver could sue the manufacturer for any defect or if the manufacturer misled the user into thinking the system was safe and didn't need human attention.
This way, if a system is unreliable, users will start blaming the system and stop using the feature or buying this car. Also, it allows manufacturers to roll-out features faster and learn faster from the real world.
If the manufacturer is directly responsible when driver assist features are engaged, but not the driver, then the manufacturer will get scared to roll anything, and progress will be much slower, and there will be more victims because the drivers think "I don't care, I'm not responsible, it's even better if the auto mode is on, even if less safe, cause I won't be responsible"
That doesn't work in general. We'd want kids to be able to use self-driving cars. And really old people, and people without a licence, or who are mentally challenged enough that they don't have legal responsibility.
> If the manufacturer is directly responsible when driver assist features are engaged, but not the driver, then the manufacturer will get scared to roll anything, [...]
Google's Waymo specifically lobbies for getting the legal liability on themselves, not on the driver or owner.
I don't think that "move fast and break things" is the approach we want for self-driving cars.
So much so, that I've read about people's fears that self-driving cars will be subject to bullying by pedestrians (and other drivers).
Why?
No
> Who's responsible?
Cruise and their insurance
* unless you're rich