Hacker News Reader

Comment by kwdc | Hacker News Reader

The irony is that this is actually a good reason to drag the car manufacturers to the table and compel them to fix it from their end.

But pragmatically, they likely don't answer the phone. Or email.

So, it is still easier to fix the road. And hope the issue doesn't resurface elsewhere and inadvertantly injure or kill too many people.

View on HN

Or simply decertify all of the vehicles that behave incorrectly from driving on public roads.

This is the kind of issue which will persist, vehicles which fail to recognize a real world situation shouldn’t get the benefit of changing the world to fit them.

An inquiry should be done into how these vehicles came to fail to recognize this road feature and the upstream cause (not just the bug by the systematic issues in the organizations which produced vehicles with this bug) should be fixed before they’re rectified to operate on roads.

I like the idea of a warning that particular models of cars are not permitted past this point for safety reasons. That by itself is enough for some frustrated driver to see the sign, take a photo and upload it to social media where those car manufacturers can be suitably laughed at and ridiculed.

The good ones would then sort out why their braking system is faulty. Perhaps it gets confused about other things in other serious ways.

I can't tell from the article whether the automatic braking happens when a sudden incline, decline or both is detected in front of the car. It was one of those three, though, that much is clear.

Do you really regard braking in front of a sudden incline/decline as ridiculous?

It seems like there was a steep hill, but the cattle gate was level, and the sudden, extreme difference in gradient is what caused the sudden braking. Although from the photos, it all looks fairly flat to me.

Yes, that's my impression too, and braking seems sensible.

From uphill: "There's something weird in front of the car, best drive very slowly". From downhill: "WTF I can't see the road any more, best drive very slowly." Both seem quite sensible to me, I don't understand why so many people on the thread seem to disagree.

"Cars have come off the road trying to cross a cattle grid after sensors mistook it for a wall and slammed on the brakes."

These grids usually lie flat on the ground and in some areas are commonplace. Seems to me these sensors are still needing work. Perhaps they see horizontal lines and panic?

Cars running off the road for no good reason is ridiculous. It is indefensible to assert this is OK. Denying it or otherwise covering it up is definiteky ridiculous.

Decertifying a self driving car does nothing considering that today the driver is legally 100% responsible for the driving. Regardless of how manufacturers sell their "fully self driving cars", they are all still a "hotdog/not hotdog" type intelligence and the tech is there just to assist the driver, not replace.

I don't believe we'll solve autonomous driving too soon without adapting the infrastructure to it rather than the other way around. At this rate we'll have truly self driving cars (not "straight line, follow the highway lane self driving") right around the time everyone starts taking most personal cars off the road to replace them with something more efficient that better suits cities of millions. But right now too many people and manufacturers are lying to themselves that self driving cars are right around the corner, and even that they're mostly here.

The solution would be to fix the disinformation surrounding self driving tech and fining the hell out of any manufacturer that doesn't make it painfully clear on every channel that the tech is very limited, mentioning also the limits of the system (e.g. types of traffic it was tested on, how many road signs it can recognize out of the total, how much of the regular driving experience do those conditions cover, how well did it do compared to an average human driver). All in "human" language.

Some blame lies with technologists, many a time I have been downvoted right here for suggesting autonomous driving is not round the corner.

As someone who only sells products using ML, I find the discussion here inorganic.

agree - some blame lies with technologists.

Maybe most of the blame.

But consider this: If a human had the same problem, who would be blamed? If several humans, then the road would be changed, just like this. The process seems sound, but maybe the changes that make it more human-friendly make it more machine-hostile.

New thinking is needed on how we retrain machines after the change, as it necessitates uniformity - the machine would benefit with fewer variations in cattle grids for example. I reckon a simple change to the road markings for all cattle grids would be enough to make them logically identical - be it paint or signs or some other physical indicator.

New infra will be a positive, we will be able replan roads and usage keep pollution in mind.

Though I don't think fully autonomous driving will be possible in near future. There are too many edge cases.

> Decertifying a self driving car does nothing considering that today the driver is legally 100% responsible for the driving.

I don't know that this is true in this case. It seems like the problem here is the auto-braking feature kicking on, perhaps disorienting the driver or causing a following car to need to panic brake+swerve. This isn't an instance of a driver becoming inattentive because they expect the computer to act for them. Rather it's an ever-present system erroneously deciding to take control of the car - analagous to a mechanical failure. What could a driver do to avoid this behavior, besides knowing to disable that system ahead of the hazard?

Decertifying a self driving car does nothing considering that today the driver is legally 100% responsible for the driving

The owner of the vehicle is also 100% responsible for making sure the vehicle is road-worthy. Decertifying the autonomous-driving feature should probably trigger a manufacturer recall, so the features can be disabled permanently (not just a via software update).

Or alternatively, most countries have a periodic vehicle check (like the MOT test in the UK), and having non-certified features enabled could be a reason for failing the test.

Actually, we hired a £32K/$43K Nissan Qashquai for a week, and came across numerous idiosyncrasies. The "best" one was when we attempted to park against a low hedge, and the car set off all sorts of warning alarms. It then refused to lock the doors, so we had to move to a different part of the car park.

The refusal to lock was probably that you'd not fully switched off the engine. I've had this on my Qashqai. The fuel-saving system that automatically turns off the engine when you're stopped and out-of-gear can be confusing. This behaviour often happens when parking but it seems you still need to press the stop-start button to "completely" turn it off in order to lock the car.

This is the first car I've owned with "modern" driver aids and it is true that they are somewhat temperamental. For example, the auto full-beam headlights get very confused in fog (which can be dangerous!), whilst I often get a warning noise when approaching sharp bends at a perfectly safe speed.

Ah, that sounds interesting - we only had it for a while so didn't get chance to get to know all the quirks... we found the motorway lane wandering feature handy, but we had great fun when we tried to lock the damn thing and didn't move away from the car quick enough: it sensed the key was nearby and "helpfully" unlocked again. After the hire period we were glad to get rid of it!

> we found the motorway lane wandering feature handy

Honest curiosity, but as someone who drives quite a lot (A, B & C licenses) I don't understand why people need help staying in their lane. Flippantly I'd suggest that piloting the car and looking at the road is the absolute bare minimum of 'due care and attention' when driving, though obviously the prevalence of this feature would suggest otherwise.

Any thoughts gratefully appreciated.

Depending on the implementation, it trains/forces you to actually signal before changing lanes (I see people getting lazy with that). But while that's good for everyone overall, it's not exactly an explanation why people buy cars with these features.

> piloting the car and looking at the road is the absolute bare minimum of 'due care and attention' when driving

I agree! In this case it was novel to see how it worked (because the facility was there and new to us) but I'm more keen on using my eyes and mirrors. I could see how it could be useful in dense fog though, given many drivers don't bother with lights...

I assumed the intention is to alert a driver who really needs to take a break. I don't see what other use it has, other than to annoy me when overtaking a cyclist.

Of course, all of these cars are chock full of computers, speakers and monitors, but none of them ever tell you what's wrong in a more intelligible way than wild screeching.

I spent half an hour outside in the cold at 2am with a borrowed Nissan Leaf. It only made a loud beeping noise when I tried to lock it. Apparently, this noise means that the trunk isn't properly closed and that it therefore won't lock the car for you.

I think a step between what we have now and full human replacement will be roads that are certified for fully automated driving. Those roads would avoid features that confuse the automation, or have some sort of passive or active assistance installed.

> This is the kind of issue which will persist, vehicles which fail to recognize a real world situation shouldn’t get the benefit of changing the world to fit them.

There are already places that regularly confuse human drivers, and those do get fixed (though usually only after someone dies). I don't see why we couldn't do the same for driving robots, especially when they get more common over time.

>Or simply decertify all of the vehicles that behave incorrectly from driving on public roads.

At least in the US we try to grandfather in old tech because the poors have enough problems without the government telling them they can no longer use their junk.

If a box was checked, a standard was met or a capacity was stated when the vehicle shipped the box remains checked more or less in perpetuity. This makes for some great internet arguments when the pearl clutching crowd tries to appeal to the authority of manufacturer statements with regard to older vehicles.

A way to do this would be to create a class of forced manufacturer recalls to disable the broken feature which if not completed would block yearly registration renewal, like the emissions checks some places require.

Indeed recently while going through the California smog check and registration process, the dealership told me I had open recalls which might block my registration. They didn’t, but clearly something like that mechanism exists.

This doesn’t unfairly harm people who already own vehicles.

> The irony is that this is actually a good reason to drag the car manufacturers to the table and compel them to fix it from their end.

The solution is having the balls to lay down good policy/regulation. There is no point forcing car companies to use a certain algorithm, for example, but it's not unreasonable to build a standard basket of problematic road signs which the car must pass - e.g. a cattle grid isn't too bad, but if a driverless car doesn't stop for a speed limit outside a school the company should have some liability.

I feel the "driver" of the driverless car should be responsible too. We are nowhere near actual full unsupervised autonomy - people should still be actively looking out for the car doing the wrong thing.

You don't get to abdicate responsibility on the road just because you bought in to some corporations marketing.

Absolutely true, although the corporate liability thing really depends on whether the engineering takes after software or aviation (to pick two widely contrasting examples).

And I notice the "phone with wheels" built by mostly software engineers chooses to use words with well known meanings from the aviation industry without a) any of the rigour behind the regulations and certifications or b) much in the way of acknowledging the differences between the challenges of an "autopilot" safely managing to traverse the airspace 37,000 feet above Hawaii, compared to a rural lane in the UK (or 280 between the valley and the city).

(While, sadly, at the same time it seems from the outside that the ACAS code got written by the same outsourced dev team that work on their website the rest of the time. :sigh: )

non autonomous cars have driver assists now that slam on the brakes when they think you’re about to crash. Statistically a huge improvement to active safety, and then there’s cattle grids.

Imagine you’re driving along in full control on an empty country road. Out of nowhere your car panics, slams in the brakes, deploys seatbelt tensioners, and flashes bright warning lights because clearly you fell asleep or something if you cant see that big wall.

Would you react? Maybe swerve coz you’re startled?

You don't get to abdicate responsibility for your vehicle's "autopilot" just because that was only corporate marketing.

Words mean things. You don't get to say they mean different things when you say them just because it helps you sell more cars. Not in the fine print, and not in a court.

There's no way a "driver" can autocorrect for a vehicle that makes an emergency stop out of the blue -- the vehicle is still stopped, and now poses a safety hazard for the surrounding traffic.

I'd go further than that. If a vehicle's automated systems caused or contributed to an accident or a road rule violation, the car manufacturer should be held responsible.

If your company's automated emergency braking, or lane assist, or adaptive cruise control - causes or contributes to a problem, someone from that company should be in court next to the driver facing judgement for the consequences.

(I'd even go so far as to suggest if the existence of a system leads to a situation where an incident occurred because of the driver's assumption that the automated system would prevent it, then the company should need to justify why the driver should be held responsible instead of the software. If your advertising claims "autopilot!", you're going to be apportioned a significant portion of liability even if your fine print says "Drivers must keep alert and ready to take control at all times" - you should be judged based on a reasonable persons expectation from the language you use to promote your vehicles, not some w4easel words in your T&Cs.)

Automation issues may fall into the category of manufacturer/developer negligence; you seem to be suggesting some sort of burden shifting.

When you say that a manufacturer should be held liable if their system '... contributes to a problem...', it seems like you're suggesting adoption of the (notoriously problematic) 'if not but for' test. This sort of test is used in negligence cases, but can be useless, as almost anything satisfies it (i.e. slow service at a coffee shop may have been a necessary condition for the timing of that exact accident).

Unfortunately, pretty much every automobile safety improvement in the last century (which collectively have saved ~millions of lives) also occasionally kills people. Airbags can propel objects into people, seatbelts can trap someone in a burning car, antilock brakes perform worse on sand, etc.

In practice, putting the responsibilty on auto manufacturers for every specific injury or death that happens due to a safety system, without somehow giving them "positive credit" for lives saved, massively delays or entirely prevents these sort of advancements from being made. We definitely need processes to verify that new features are actually functional, typically through recalls and product liability suits, but simply putting blame on manufacturers for all accidents is a terrible idea.

> but if a driverless car doesn't stop for a speed limit outside a school the company should have some liability.

This is a particular challenge in California because our school zone signs specify that the limit only applies "when children are present."

How are you meant to know whether children are present in time if you're not doing the lower speed limit?

The idea is that you would be going slow enough already (and if not, slow down). Then, if there’s no children, you can speed up.

It weirded me out when I moved here, but in practice, it's enforced like anywhere else: at the beginning and end of school days.

Since a large fraction of kids are driven to school (at least in southern California), you know that this is happening because of the massive traffic jam near the school.

Illinois has these "no turn on red when pedestrians present" signs and I have no clue how thats supposed to help anyone because pedestrians usually get hit because they aren't seen by the car.

In Germany, when a speed limit of 30kmh is introduced because of a school, it is usually on applicable for school days and hours (e.g. Mo-Fri 8-18) only. After school hours or non school days you are allowed to drive at normal speed of 50kmh.

> it's not unreasonable to build a standard basket of problematic road signs which the car must pass

I'm wondering how you would get around metaparameter optimization here, or the fact that it's too easy to optimize to a known test procedure rather than the real world?

For unusual fixed features like this, perhaps car manufacturers should start putting exceptions into their navigation databases, rather than tweaking their automatic detection. You generally don't want to be driving over cattle grids, with their sudden gradient changes, at high speed anyway. So tell the car to slow down as it approaches (or warn the driver loudly to slow down), and briefly disable automatic braking. If it's a rare event, it'll have the drivers' attention.

I see the future of self-driving cars to be one where highway authorities map and sign features in a way that is designed for self-driving cars to be able to parse, rather than requiring them to heuristically sense as much as they do today. This would be a step down the path that's inevitable anyway.

Giant QR codes on highway signs? Haha, an attack by spoofing them would be interesting.

Next trick: QR codes with cryptographic signatures, and a huge traffic jam when the traffic authorities forget to renew their certificate...

Having a database of known trouble spots would certainly serve as defense in depth, but would only be secondary.

> But pragmatically, they likely don't answer the phone. Or email.

> So, it is still easier to

... hold them responsible in court for damage/injury/deaths caused by the actions their vehicles take which are not under control of the driver.

I prefer proactive policy. As much as I would have liked to see more people go to jail for the 2009 financial crisis, for example, I’d prefer to see smarter regulation to have prevented it in the first place

Regulations are too complex to get right like that. You need to figure out how to enforce following the spirit of them not the letter or you will forever be seeing more crises that you didn't foresee.

I think that’s the ideal, but tends to break down in practice. The problem with expecting the spirit of the law to be the compass is that it opens up a lot for interpretation. The end result is that it incentivizes gamesmanship where the organization with the best lawyers win

While I agree - I reckon it's far past time that we pierce the outdated "corporate personhood" idea, and let Tesla/Uber/Apple/Waymo et al. know that society has chosen to hold their "corporate personhood" responsible in the same way people are held responsible.

If your corporation (via it's self driving software) kills someone, the corporation will "go to jail", a C level exec does the jailtime and the entire company will be unable to carry on any of it's normal business for the duration of the sentence (while still being held responsible for it's obligations to staff/suppliers/cerditors). If that self driving software is judged to be liable for a traffic infringement, the corporation will pay an amount equal to the percentage of income that a human driver would have paid. If it's an infringement with a ~$400 fine, the corporation should pay ~0.5% of their annual income - the same sort of penalty as "an average salary" person.

A couple of $20mil fines and a 6 month total company shutdown would be very "proactive" in encouraging Tesla to get Elon to shut the fuck up with his big bold claims that cannot be backed up. If Uber had been slapped with the "corporate personhood" equivalent of whatever penalty a human driver who'd killed that person crossing the road with a bicycle they'd be far more circumspect about using minimum wage contractors as "safety drivers" in their prototypes.

(I can dream, right?)

At a certain scale, it's inevitable that even a responsible company will screw something up. Shutting the company down when that happens is tantamount to banning companies above a certain size

Taken to the extreme, what you're suggesting amounts to "large corporations should be allowed to kill people occasionally".

I can see a wide spectrum between my (admittedly over punitive towards corporations) ideas and this (somewhat intentional misinterpreted) idea of yours. But I do honestly think Uber and Tesla have evaded morally deserved punishments by blaming customers or contract staff who were pretty much doing what everybody would expect to be able to do using the technology as described.

"it's inevitable that even a responsible _person_ will screw something up" - but that doesn't mean we give them a zero consequence break if they kill or injure someone. (We do, however, distinguish between things like murder, manslaughter, and negligence-causing-death. And penalise differently, based at least partly on the ability of the punishment to discourage other people from doing similar things. Perhaps Uber was only guilty of negligence-causing-death for using minimum wage contractors as safety drivers. Perhaps Tesla was guilty of manslaughter by advertising as "autopilot" a thing that drives full speed into offramp dividers?)

We have already accepted this in other areas where low probability meets large applicability. We know probability is non-zero and given a large enough sample size there is an expected adverse outcome quite a bit higher than zero. E.g., a more trivial example is that the allowable number of insect parts in food is greater than zero because it’s impossible to mass produce food and never have contamination.

We shut down Boeing. As per your argument their planes should still be flying.

Would the corollary be that autonomous cars should have a certification for self-driving, similar to the FAA certification? Possibly an extension of their car safety rating?

As a programmer, I find it comforting to hear that people are prepared to terraform in order to cancel out my bugs.

While the question of who pays for it is important, we should be focusing on the best way to get safe driving.

If car sensors and algorithms can get to 99.99% of road features, then changing the remaining 0.01% to work with modern car technologies seems like a reasonable use of resource. No algorithm is ever going to be completely perfect.

I'm more worried about the roads that can't be changed.

What happens if there are bugs or we find code being not up to the mark? Should all the models using that version be disabled till OTA updates are rolled out.

For reference bugs in Boeing's plane resulted in the whole fleet being grounded.

I don't see a problem with autonomous vehicles having a manual control mode, unless we're talking about a scenario some decades in the future where you don't actually need a driver's license for an autonomous vehicle.

In the small, it's dramatically more cost effective to fix the road. Ignoring cost to develop and test, just the cost to deploy a fix to all the affected cars would probably be a lot more than £70,000.

In the large, it's always a shame to fix a good test case.

Reply on news.ycombinator.com