Hacker News Reader

I've worked on CNC mills and lathes, design of the logic and the hardware driving the servos as well as some fuel estimation software for airplanes.

The degree of care that went into the e-stop design of the lathe and mill controller and the number of checks and tests of the fuel estimation software are stuff that I'm (hopefully justified) proud of, that's not stuff to cut any corners with. In both cases that took me longer than Hotz claims to have spent on creating the whole self driving car code, there is no way you can do this safely with that mentality. It is absolutely irresponsible to take that attitude towards systems like this, life is not a video game with 3 more lives if you lose this one (or cause others to lose theirs).

This. There is a huge amount of safety evaluations, precautions, training and failsafes we have to go through just for using a Class 4 laser at work in a controlled environment, where the worst possible outcome is permanently blinding someone working with it.

For a machine with the potential to easily maim and kill, to be completely autonomously operated in a chaotic environment where it interacts not just with other computer controlled machines but also with humans, even children, I honestly hope that the amount of safety precautions required will grow very quickly.

Google I'm totally cool with. They've been at this a long time and still don't seem very close to actually releasing a product.

I like Tesla's and what Elon's accomplished has been impressive but he seems hellbent on "winning the race" to release a self driving car and that worries me.

It's a good concern to have, but I also worry about waiting too long here. At the end of the day, there's going to be problems that the testing would have never found regardless of how much testing there is, and we need to be psychologically prepared for that.

It took airplanes a long time to become safe (through a long and scary process), and now they're safer than my front porch. Whatever the outcome, self driving cars will very likely be safer than human drivers, and that means more lives are saved than lost, even with the occasional horrific problem. It is entirely plausible that self driving cars are already safer than human ones.

I wish it were possible to buy a Tesla without any autopilot features except basic cruise control, because I know how damnably difficult the edge cases are for a self-driving car, and I'm perfectly capable of driving a car manually, thank you very much. Tesla's cars are so good I'd almost be willing to pay more for one without autopilot.

Almost.

While driving, you have to do a double-pull on the lever to activate the autopilot, a single pull just activates the basic cruise control. So you can still buy one and just never activate it.

Alternatively, earlier editions of Model S shipped without autopilot, which was then sold as an aftermarket upgrade as late as this spring (when autopilot was an optional $2,500 package on a new Model S).

Fair enough re: the older Teslas. As to the new ones, even if I never activate autopilot I'm still driving around in a massive autopilot data collection engine for Tesla Corp, which bugs me.

Reminds me of wanting to buy a TV without all the "smart" stuff. In that situation modularity is preferred by most people with at least minimal tech-inclination. I actually encountered some cases of manufacturers charging more for a lesser, non-smart panel.

A car is a bit more complicated to leave stuff out I am sure and the volumes far different. It would likely require at least more resources to have separate parts of lines for some of that as well as different check procedures as so on. But like you said, paying more could help offset that, though it's a gamble for the manufacturer.

I'm more in the camp of please do not release anything driving a car until you guys can figure out how to secure basic software, keep websites running, parse input properly, and so on. In other words, I'd rather them wait decades if that's what it takes.

Personally, I'd rather be killed by a person rather than a person's dumb choice trusting technology. What concerns me in all these situations is that I did not at any point agree or vote to allow someone in a self-driving car to drive on the same road as me. Until they can mathematically prove 100% it is safe in all conditions and situations, I'm not buying things like "driver assist" if a big part of the justification is that AI is better than a human driver (if so, then why assist?). Others may disagree, so be it - I'm a weird guy. I also hate cruise control. And fat-free ice cream.

If 10,000 people were killed on the way to making driverless cars safe, and this wild west approach ended up getting them into mass market use a few years earlier, it would save the lives of thousands of people overall.

Is this a bad thing? Is it not justified by utilitarianism?

That would kill the driverless car concept.

See, we already have cars, it's not like we're going from 'horses and buggies' to cars, it is going from one good solution to an even better one and even though there is a risk at being stuck at the present local maximum for a little longer going down in order to find the peak of this particular mount improbable would not be an acceptable path for the majority of the public.

And without having a time machine or a functional crystal ball you won't know if it is 10K, 100K or a few million people that get killed before you get it right so the bar has been set at 'it needs to be better than the present level before it is acceptable'.

> If 10,000 people were killed on the way to making driverless cars safe, and this wild west approach ended up getting them into mass market use a few years earlier, it would save the lives of thousands of people overall.

This is a perfect example of the biggest and IMO most compelling critique of utilitarianism -- namely, that it's too susceptible to bullshitting.

People decide what they want to do, then completely invent (or, in more sophisticated cases, carefully fabricate) numbers and time-frames in order to hide their unethical opportunism behind the facade of objective moral assessment.

This is especially problematic when you have to estimate things that are pretty much impossible to estimate without completely bullshitting. For example, "number of people killed by not following certain safety standards", or "impact on time-to-completing of some particular safety standard". We're talking about a novel technology, so there's no way anyone has even remotely high confidence in a sufficiently bounded estimate of these numbers. It's all just bullshitting.

> Is thus a bad thing? Is it not justified by utilitarianism?

These are two VERY different questions.

Lots of things are susceptible to misuse. That doesn't invalidate them unless they are in fact misused.

Moreover I don't see how you can conclude on a whim what's possible to estimate and what is not. Given these things take expertise in the relevant field and more than just casual analysis.

Finally, it's not like there is a small margin of error. Can you even really comprehend 30,000 deaths every single year caused by conventional cars and drivers? This is fricking 9/11 times ten happening every year, and you're not even willing to consider that speeding up r&d might save a few lives?

I would absolutely consider speeding up R&D to save that number of lives. I'm 100% behind self-driving cars. I think self-driving cars will eventually reduce the number of road deaths significantly.

I don't think the case has been made yet that today's tech will reduce the number of road deaths. I don't think it's unreasonable to be skeptical of the claim "this works great, trust me".

I'm very skeptical from no-name nobodys like Hotz running comma-ai and very non-skeptical of companies like Tesla.

> Moreover I don't see how you can conclude on a whim what's possible to estimate and what is not

Yes, exactly. Like in so many cases, coming up with good values for the parameters is often the most difficult part of the analysis... that was exactly my point!

> Given these things take expertise in the relevant field and more than just casual analysis.

Except in this case, there are a lot of experts, and they all seem to be taking roughly the same approach.

> Finally, it's not like there is a small margin of error

It's not clear to me, at this point, what your assumptions even are. What percentage of those deaths would be prevented by self-driving technology? How many new types of accidents would be caused by self-driving technology? By what percentage would that number decrease if the regulatory environment were more friendly? And what's your basis for these numbers?

Also, it's perhaps telling that there really hasn't been a single example of regulators preventing self-driving up until this one, which is rather extreme. Google, Uber, and the scattering of startups and car companies have all taken a more cautious road to deployment of entirely their own making.

> and you're not even willing to consider that speeding up r&d might save a few lives?

There's a big difference between speeding up r&d and rushing unproven tech to market.

So what do you propose as a guiding principle in place of utilitarianism?

Your scenario would save lives, but we have no way of knowing if we are in that scenario.

If you know that after ten thousand deaths, the technology will be good enough to start saving more lives every year, then your utilitarian argument works. The problem is that you can't possibly ever know that in advance. It's equally plausible that the last 0.001% will take decades to solve, or even remain out of reach indefinitely. Then you just threw away ten thousand lives for nothing.

This is why the burden of proof needs to be on the new system, and why the precautionary principle applies.

i understand where you're coming from, but there is a counter argument to what you're saying, which is that although there may be some dangers to things like the comma one and tesla autopilot, their release may save more lives than they edanger, having a net positive to humans. humans are pretty bad at driving

This is an ends justify the means argument. How many lives have been saved TO DATE with self driving cars? How many lives will be saved IN THE FUTURE with self driving cars?

What is the acceptable casualty rate to get from where we are now to where you think we will be in the future. Is your family an acceptable casualty? If not, then why is mine?

"Is your family an acceptable casualty?" is a pointless question. It can be applied equally well to the opposite argument: what is the acceptable casualty rate for not advancing safety in this way? Is your family an acceptable casualty?

Continuing on with the status quo is just as much of an action and a choice as pushing forward with automation. Both choices are going to get people killed. We can either evaluate the choices and choose the one that seems better, or we can ignore the question and possibly have more people die than is necessary, but there's no option where your family isn't at risk.

This argument is a red herring. We know the current casualty rate and have accepted it. Nobody gets into a car with the illusion that it's 100% safe.

And if technology reduces that casualty rate, why does the fact that it's new suddenly make it unacceptable?

I didn't say that. My question was: what is the casualty rate currently and what is the casualty rate in the self-driving car future? I am asking what you think this reduction actually is and how high of a price you're willing to pay to see that reduction. I am specifically asking what you think the reduction is so that we can have a discussion on 1) whether that is a realistic reduction and 2) whether that is, in turn, worth the price. It has nothing to do with whether or not it's new.

Is that reduction a result of self driving or advances in other safety (airbags, seat-belts, crumple zones)?

It is clear that your mind is already made up and that you are a self driving car evangelist. Please endeavor to approach these debates with an open mind and in good faith.

My mind is made up only because I'm pretty sure the casualty rate will be sharply lower. Humans are horrible at driving.

As for your question, you basically destroyed the conversation when you asked "Is your family an acceptable casualty?" If you just wanted numbers, you shouldn't have made it personal like that. That's all my point is here.

> As for your question, you basically destroyed the conversation when you asked "Is your family an acceptable casualty?" If you just wanted numbers, you shouldn't have made it personal like that. That's all my point is here.

This is a really fair point but I also don't think it's a bad thing to say this. I think people distance themselves from possible problems by calling things a numbers game but they don't factor in the fact that they could be part of those numbers.

If I say that 27000 people per year die from listening to Mambo Number 5 then I say that we can lower that to 15 if we remix it to take out the deadly frequency but we will have to test it on 20 people to find which is the deadly frequency for sure. That sounds great. A couple people might die but what's that to save 27000/year?

Well, take that and say we will have to test it on 20 people and 5 of those will be your family. That makes the decision a lot harder. Logically it's exactly the same but emotionally instead of "people" it's 15 "people" and 5 of your family.

So, it's important to mention it could be your family because if someone was alright with it because they are looking at the pure numbers, they should be brought back to the reality that we all actually live in a little.

I agree with the effects but I disagree with the desirability! We should be making these choices rationally. Taking an action which causes some people to die but saves a more overall is typically a good thing. Asking, "what if it's your family?" just makes it harder to make the correct choice, and that makes it more likely for your family to be a casualty.

I guess this all boils down to what you think of the Trolley Problem. I'm one of those who say, yes, of course you pull the lever if you have no way to save everybody, it's obviously better.

> Continuing on with the status quo is just as much of an action and a choice as pushing forward with automation

Of course, but public perception is, you know... not largely logic-oriented. Whether or not you personally find @abduhl's argument convincing, there's still that emotional hurdle you'd have to clear to convince significant numbers of people to sign up.

Of course. But acknowledging and working with that illogical public understanding of things doesn't mean I can't argue against it when I encounter it online.

I don't necessarily disagree with you but one thing that I think a lot of people are seeing is you are relinquishing control of your car to a computer. When I think of computers I often think of things that I CAN MAKE WORK but that aren't inherently trustworthy. Sure they're great for things like math and stuff but when I try to run a new piece of software I expect crashes and they generally happen. NOT ALL SOFTWARE. But a good bit of first gen software.

So do I really want untested first gen software released without extreme safety standards and heavy testing? No. Do I think the net result of that happening would be a safer driving experience? No. Give me the iOS 9 or so of self driving car software or the unix. Not the Amazon Fire phone.

I totally agree on this. Putting my life in the hands of software makes me nervous. I've seen how software is made. But I do it anyway....

I'm not sure if it's entirely relevant here, since the comment above presumes that these systems are safer than humans, and the reply doesn't seem to be questioning that.

I think it's very likely that these systems will rapidly become far better than human drivers, and probably already are in the domains where they're meant to be used. But we definitely can't just take that on faith and hope it's being done properly.

To estimate your questions, probably no lives have been saved so far through self driving, possibly one extra guy killed in the Telsa crash which you could say was maybe half the systems fault and half the driver not following instructions.

In the future, 1.2m people die a year so say self driving reduces that to 0.2m/year, that's a million lives saved per year.

You family is thousands of times more likely to die in a conventional motor accident than due to a self driving vehicle. If unnecessary red tape slows things down and causes 100k extra deaths is that ok? Even if someone you know is amongst the 100k?

That said there's an argument system like Hotz's should be tested first. That's probably not going to slow things that much.

If unnecessary red tape slows things down and causes 100k extra deaths is that ok?

That's an imaginary scenario. You have no idea if that's the way it plays out. Let me give you another scenario: the first self-driving cars kill a bunch of people, and no one wants to touch them for years after they finally become safe; delaying the adoption of cars, and causing millions of additional people to die.

Very unlikely. Self-driving cars are potentially enormously profitable; with such profitable products I think you'll find that the opinion of an outraged public matters somewhat less than it customarily does.

As a great man once said, "Not taking any action is also an action."

That's even MORE of a reason for the NHTSA to ask these questions. If the first-to-market autopilot system is too dangerous, that will destroy the public's trust in any future autopilot system, no matter how much of a net benefit it provides.

It's like how IUDs (birth control device) are disproportionately unpopular in the United States despite many benefits - all because of one particular brand that caused infections with horrible repercussions:

http://www.motherjones.com/blue-marble/2012/09/why-are-iuds-...

I hate this bullshit argument that 'humans are pretty bad at driving.' Yes there's a wide distribution of driving skill levels, and this is dependent on the cars people drive and their maintenance records. But please don't lump me in the same bag as obviously shitty drivers, of which I've seen scores of.