Hacker News Reader

Comment by inamberclad | Hacker News Reader

I know this article is satire, but it certainly strikes a chord. Putting big metal robots capable of moving at highway speed in close proximity to squishy humans is going to cause issues. It might be the human's fault. It might be the vehicle's fault. It might be completely unrelated.

However, there are industries like aviation that take these issues seriously. They're investigated, reported, and learned from. We need to do the same for autonomous vehicles if they're actually going to be safe. This means we need an open, independent body outside of industry control, like the NTSB or NHTSA that can produce in depth, transparent, and publicly available reports that manufacturers can't bury or buy away.

On the other side, we, the (future) passengers need to be conscious of these safety issues. We need to hold companies accountable for fixing flaws in autonomous vehicles. Airplane accidents can be spectacularly big, which captivates the public. I'm worried that car accidents will just be too small for enough people to care, and therefore we will continue to have a trickle of fatalities in every city and we will never be free from cars.

View on HN

~40k people die in the US from human caused car accidents every year (~12.9 deaths per 100,000 people, ~1.37 deaths per 100 million miles traveled), and we're getting squeamish about the robot that never gets tired, drunk, or inattentive?

If you want to be free from cars, build walkable and carless cities people can afford and want to move to. We can then discuss sunsetting mass automobile infra (which is very unlikely, although infra spending can be weighted to more efficient urban development, mass transit, bicycle infra, etc vs automobile centric systems). Until then, the need for autonomous vehicles is an inconvenient truth.

> This means we need an open, independent body outside of industry control, like the NTSB or NHTSA that can produce in depth, transparent, and publicly available reports that manufacturers can't bury or buy away.

Strongly agree.

> 40k people

You're jumping to a quantitative comparison because that's simpler and more obvious, but the right way to think about it is qualitatively.

We have over half a century of understanding of modern human driving patterns. We know the ceiling on the number of accidents/deaths that humans will cause. We know that if it changes, it will change slowly.

AI works differently. An update to code or models wipes out all expectations we've learned from prior versions.

In short, the entire fleet of autonomous vehicles can change behavior overnight with a software update and become murder tanks.

Nothing will ever happen to all human drivers at once. Some small percentage will be drunk at any given time. We'll never all wake up with a new drunk version of ourselves at the same time.

I don't mean to say that autonomous technology could never work in principal. But we know that this SV in practice has too much hubris and too little oversight to deploy sensitive products responsibly, or with attention to any values above profit and growth.

That's what regulation is for, we have a good example in the FAA here, now we'll need something like the FAVA (Federal Autonomous Vehicle Administration) to design regulation for autonomous cars.

We have a lot of precedent at this point on how to handle new ambiguous human advancements. Typically it goes, discovery, wild west, blood, regulation, standard part of society. Rinse and repeat.

Thank you, regulation, I forgot to mention it. That's another pillar of the argument that we know from history that we can't rely on our current structures regarding how we receive SV's rate and style of "innovation".

That's not really true though. Tesla for instance is regulated like a car manufacturer, not an sv app developer. This is bit of a false equivalence you're putting forth.

Have you ever dealt with automotive regulation? There are standards, sure, but most of the actual verification in the US basically amounts to "trust me bro" until there's overwhelming evidence of an active, fatal safety flaw. It's retroactive and increasingly inadequate to how modern vehicles are designed.

I have not, but I was assuming it was better than regulation of apps, of which there is almost no regulation. Sad to hear it may be otherwise. I was hoping it had the same track record as aviation regulation which is pretty dang good.

We could also simply just not update them all at once. This is just baseless fear mongering. Obviously everyone agrees there's more engineering rigor that should go into these things than your run of the mill CRUD app from a seed stage startup.

Require canaries of substantially significant changes through regulatory mechanisms and approval. Regulators should get access to the same telemetry the manufacturer has.

>substantially significant changes

any change is significant, even the smallest, not just due to the possible bugs, but also because of the potential risks/side effects that can occur during the update + patching process of the car. regulation will need to have automated driving go back to waterfall style deployments that get applied by a licensed mechanic.

until such regulation is in place, i recommend not driving during the last/first weeks of Q1 Q2 Q3 Q4, as dev teams will be rushing OTA patches in order to complete their projects.

Softwares may have a lot of undiscovered bugs, but it's still way more dependable than humans. And, bugs in softwares are also written by humans. So, imo, we just transform the risk of human driver to human programmers while transforming to autonomous vehicles.

> If you want to be free from cars, build walkable and carless cities people can afford and want to move to.

Oh man would I love that, but our legal system makes this nearly impossible, especially when combined with car culture.

It's a guaranteed way to reduce pollution, improve happiness and health, save lives, etc etc etc. But it is easier to solve computer vision than go get cities to ban cars.

There are many people like me that see city life as almost full dystopian. My health improves if I don't live around many people and if I have room for a large garden for food, both human and pollinators. I am most happy if light pollution is almost non-extant (so are nocturnal and crepuscular critters).

I understand the attractiveness of the walk-able city to some. I hope you (and other of your kind) can come to accept that there are others that have different needs for living space.

> I hope you (and other of your kind) can come to accept that there are others that have different needs for living space.

Is this some sort of weird projection? My preferred way of life is legally banned, and you want to keep it banned, so you assume I want to ban your way of life?

Furthermore, the more of my lifestyle, which is compact, the easier it is to maintain empty space that you prefer.

Clearly you view me as an enemy, but I do not view you as one. I do not wish to make your way of life illegal, and only expressed a view that my preferred way of life should be legal.

I am flabbergasted by your comment, honestly.

> Is this some sort of weird projection? My preferred way of life is legally banned, and you want to keep it banned, so you assume I want to ban your way of life?

It is a comment on a lack of empathy for different way of living. I don't want to ban your preferred way of living as long as it does not pollute my night sky with light, block my sun, or keep my from traveling to where I want, when I want.

> Clearly you view me as an enemy, but I do not view you as one. I do not wish to make your way of life illegal, and only expressed a view that my preferred way of life should be legal.

No, I do not see you as an enemy either. this is a conversation with lots of nuances that are lost. I believe the loss of nuance has led people pro and con on the 15 min city model to be hostile to the opposing view. I'm trying to find a way to say, I support you having your 15 minute model, I hope you will support my not wanting it or at the least, don't force it on me.

I am assuming that when you say your way of life is legally banned, you are referring to zoning regulations. Changing zoning regs means you are going to need to become part of your local government. I suggest you go to planning board meetings. Listen a lot, ask polite, questions. Remember these people are not getting paid enough for all the shit they get thrown at them. No matter what decision they make, somebody will be in their face about it. Please remember, it I'm not your enemy, most of them are not either.

Once they get to know you, volunteering for subcommittees will give you experience for dealing with town/city government. Then you can have a shot at a planning board seat and have the knowledge to make effective change.

zoning laws only change from the inside. so, get on the inside, and make change happen!

Who is the one with the lack of empathy here?

You assumed that I had zero empathy and wanted to get rid of your way of life, on what evidence? Merely because I could conceive of a lifestyle that didn't depend on cars?

Your comments on zoning are incredibly naive. It's not just zoning, it's building code like parking requirements, etc. It's the entire legal structure, culture, and attitude of those with power. I am involved in multiple different dues paying organizations that work to change zoning (not for my dream of a car-free neighborhood, but other reasons), and changing zoning is not the work of a motivated individual, it's the work of teams of highly organized volunteers working to provide some sort of alternative to the hoards of angry residents that harass city employees, yell, and become the most unpleasant people at the mere though of more neighbors.

The creation of a car-free neighborhood is a legal effort that is an order of magnitude more difficult than getting a zoning change. There are so many roadblocks.

That's why it's easier to solve computer vision than it is to create a car free neighborhood. The legal system is welcoming to entirely new cars with computer drivers. It is not yet ready to accommodate car-free neighborhoods.

I tried to describe the general attitude expressed by the 15 minute city advocates. Maybe I haven't tuned my wording right yet so let me be blunt. It's not about you. Stop personalizing it.

on zoning: parking Requirements: defined by zoning regulations

legal structure: built to enforce existing zoning regulations

General Building code: 99% driven by safety and good engineering practice. Usually most of the building codes come from standard-setting organizations such as the National Fire Prevention Association. https://www.nfpa.org/Codes-and-Standards. Many municipalities accept these regulations as is because some have been sued for not following published building codes when something fails and somebody dies.

attitude of those with power: get into the power structure in the way they'll accept, get elected and then you have the power.

Need for teams of people: you underestimate the impact of very small number of people can have on regulation at the municipal level.

Angry horde: that will not change until you shift the Overton window on their perception in a nonconfrontational way. this is why I advocate for starting with rebuilding the urban core with high density living space. You can sell it to them much more easily because:

it doesn't threaten their environment, it wouldn't increase car traffic, it wouldn't significantly increase load on the city core, increase the development of more entertainment offerings in the core.

for me, where the biggest wins would be getting rid of the crappy old slumlord housing. It breaks my heart to see kids with almost no exposure to nature living in housing surrounded by asphalt. That's another thing about bulk housing, the green spaces usually manicured patches of grass that you can't play on. This should be big patches of native plants and places where kids can get dirty on clean soil. Maybe even an embedded community garden. Most importantly, don't light the place up like it was Times Square.

In a nearby city, there was a significant disaster in trying to increase housing density. They chose to go the ADU route. Because it violated all of the rules I laid out above, there was a massive citizen uprising and rejection of this plan.

Your bigger barrier is going to be commercial groups. I've seen astronomy groups trying to get in place Light pollution control regulation at the state level. A single local lighting manufacturer has managed to spike that regulation every single time for the past 15 years. On the other hand, individuals have been successful at putting in light pollution control measures into local building codes.

Hear me clearly. Single individuals have made these changes happen. How? By getting to know the people in the planning board, being polite, nonconfrontational, presenting clear rational information and most importantly not being dogmatic rigid advocates. Individuals or small (2-3) Groups of individuals have been successful in changing conservation Board attitudes and regulations that accepted by the planning board.

I wrote what I did because of practical real-world examples. my experience is very different from what you have described so far. since your approach isn't working, would you consider a different approach?

Putting high density in the urban core does frequently and constantly threaten the suburbs outside of it.

Suburban voters generally want the urban core to both be easily drivable to, but also full of life and culture. This is a contradiction that city officials are always trying to compromise on, instead of simply acting according to the wishes of the urbanites living there.

I've not seen the core threatening suburbia unless the core is trying to expand and take over suburbia. In That case I say demolish the core and build denser in the core.

I understand your point your point about suburbanites wanting the best of both worlds. You are right at city leaders should take care of their own populations first and not try to accommodate suburbanites beyond making it easy to park their cars on the outskirts of the city and take public transit in.

regarding city life and culture, my partner and I used to live near a major city and it was so stressful driving 5 miles to events in the city or taking slower public transit that we stopped going to city events and looked outward to events outside of the major city.

When I lived by myself and fairly close to the city, I would drive part way and and then take public transit to an outdoor concert. I stopped going because of the size of crowds, people not being quiet so others can listen to the music, and security was obnoxious and all for show. Same thing with a Shakespeare in the Park event. Too much noise, couldn't hear the actors and it was just not worth the effort.

we now live in a much quieter, less people dense place and for us it's a much better quality of life.

In my city, Toronto, this is a constant issue. Every election is effectively a political struggle between the suburbs and the urban core.

> If you want to be free from cars, build walkable and carless cities people can afford and want to move to.

"It will only encourage the lower classes to move about, and this is certainly too big of a step." -- The Duke of Wellington, on railroads.

> robot that never gets tired, drunk, or inattentive?

it also never gets bug free

Try to find me a bug-free human. Now find a statistically noticeable number of them and you might have an argument.

"~40k people die in the US from human caused car accidents every year (~12.9 deaths per 100,000 people, ~1.37 deaths per 100 million miles traveled), and we're getting squeamish about the robot that never gets tired, drunk, or inattentive?"

And never is held responsible, never compensates its victims and never gets punished.

It's anybody's guess. No one knows who will be held responsible. I find it difficult to imagine a so-called "tech" company accepting responsibility for damages, injury or death caused by software, but who knows.

In the one case so far in AZ, Uber was not held criminally liable. Instead an Uber driver plead guilty to negligent homicide. Uber quickly settled the civil case with the victim's family. The amount of the settlment is not public information.

https://www.reuters.com/business/autos-transportation/backup...

https://money.cnn.com/2018/03/29/technology/uber-fatal-crash...

If anything this will turn out to be the opposite of the truth: People hit by autonomous cars will have more to go after in court than those hit by individuals, many of whom don't have adequate coverage or savings to compensate victims. The minimum "bodily harm" insurance coverage in California is only $15,000.

"Human drivers kill a lot of people" is true "AI will probably kill fewer people" is true but probably is doing a lot of work when you turn that into "no reason to slow down AI full speed ahead".

In fact that is exactly the point of the article, we shouldn't fall into the trap of not worrying about deaths for AI just because we have deaths from humans.

After all part of the problem with human deaths is we consider it normal. If we can avoid that trap with AI we can ensure we do better.

The "legacy automakers" that took the meticulous approach, gradually rolling out boring sensor suites with boring software in controlled conditions, are running circles around the reckless upstarts who kill people on a regular basis.

We do not need to sacrifice safety for progress. We don't need mad scientists in the real world. Sanity leads to better results. It doesn't pump the stock price all that well, though.

> > However, there are industries like aviation that take these issues seriously. They're investigated, reported, and learned from

If everything were to be as safe as aviation the whole world would come to a screeching halt standstill.

And besides they do it to protect the 200M plane not the passengers because those same passengers enter in the subway or a mall or a stadium or a train 15 mins after they disembark from the 200M plane, and they could bring (or face someone with) an assault rifle without facing any screening or x-rays or even an ID check

I don’t like self driving cars but if the bar to plow ahead with deployment of a new tech is commercial aviation post 9/11 then we can call the end of technological improvement already.

There's a lot of speculation here: aviation manages to be simultaneously extremely safe and a cornerstone of the global economy. It's unclear why other modes of transportation can't do the same; more precisely, it's unclear why private cars get put in the "acceptably dangerous" bucket when just about everything else is astoundingly safe (trains, planes, public & private buses, etc.).

(Independently: the bigger reason the FAA and airlines take security seriously is because consumers demand that they do, not just because the planes themselves are expensive. Humans are notoriously bad at estimating risk.)

Cars are extremely safe. They're driven by barely trained humans which causes the problems.

Give only a small fraction of people the ability to drive after significant training, and make driving their full time job, and you'll probably see crashes decline significantly.

This is why I think we should never have brought in unleaded gasoline. Flying still uses leaded gasoline because regulations are written in blood. Pretty insane that we just let car manufacturers switch to this when a properly regulated transportation system would never consider such a high risk operation.

It’s probably best we return to the tried, true, and most importantly safe system: leaded gasoline.

This is clearly meant to be sarcasm, but I think you've got your facts wrong: commercial jet airplanes don't use leaded gasoline. They use Jet-A and similar formulations, which are essentially just kerosene[1] and contain no lead[2].

(I think you're thinking of avgas, which is used by piston-driven engines. These don't appear much in commercial aviation anymore.)

[1]: https://en.wikipedia.org/wiki/Jet_fuel

[2]: https://www.faa.gov/newsroom/leaded-aviation-fuel-and-enviro...

Regulated by the same organization for good reason. Personal aircraft to line up with personal cars.

> > Aviation manages to be simultaneously extremely safe and a cornerstone of the global economy

Because physics gives it such a huge advantage that it allows aviation to sit on its lead and waste some speed to pursue safety and still beat trains even where trains are very efficient and dense such as Europe. It remains to be seen what will happen now that Japan is about to enter a phase of quasi-jet speed type trains.

The fact that every time you board on a plane you are treated like a criminal who is about to enter prision is unacceptable and would have never been unacceptable in the 1980s regardless of what would have happened terrorism-wise. Matter of fact the 80s were the peak era of terrorism

> > consumers demand that they do....Humans are notoriously bad at estimating risk

Of course....and of course. A rational person's preoccupation should be reducing the chances to die from any cause, because once you are dead it's not that important what was the cause, doesn't it? It's very irrational to put heavy pressure on airlines and airports when 15 minutes after you exit from the terminal you enter in a mall and some neo-Nazi can absolutely blow up your brain with an assault rifle they bought without having to do not even an ID check, and I mean blow you up to the tune that your family members cannot even hold an open casket cerimony.

> > it's unclear why private cars get put in the "acceptably dangerous"

Because driving is a very powerful action, you turn the wheel right and the car goes right, turn left and it goes left, accelerate and it will speed up, break and it will come to a halt. It makes people feel very powerful and in control, people feel in control not only conceptually but in a limbic way, so they demand way less safety standards.

Trains don't give that feeling because people aren't driving but people can at least wrap their minds around trains as vehicles, commercial passenger planes look so big and feel so gross and non suitable for flight that people are very suspicious of them. Nobody is afraid of cars, trains or ships, but fear of flight is one of the most common fears. Commercial passenger plaens after 9/11 forget about it you get to the aforementioned current day scenario where you are treated like a criminal every time you are boarding a plane

> If everything were to be as safe as aviation the whole world would come to a screeching halt standstill.

Trains are just about as safe as planes. Busses too. It's really just cars that are a major outlier. [1]

[1] https://injuryfacts.nsc.org/home-and-community/safety-topics...

Not everything, just things that can kill hundreds to thousands of people if they go wrong. That is safety engineering 101; identify the important and critical parts and do those parts better and more carefully.

You do not need to uniformly improve all elements to the same high standards as long as you can isolate them from the safety-critical path. This is a necessity because, as you point out, doing everything to the high standards is economically infeasible. But doing everything to economical standards is mass murder. No one standard works for the entire system, so you must isolate and apply the correct amount of effort for each component.

This is a very weird concept for most software developers because most software is a big mushy ball of spaghetti where it is impossible to isolate or even identify the critical path. Luckily, those individuals largely do not develop safety-critical systems, so the lack of knowledge of basic safety processes does not hinder their deployments or result in a danger to public safety.

What is problematic is when those people start doing safety critical systems. That is how you get the ideas being satirized by the article. Acting as if there is nothing in between total recklessness and impossibly perfect safety, so there is no choice but criminal recklessness. Basic industry-standard processes are, in fact, like 6 orders of magnitude safer during development and increase development velocity toward a safe, working product.

Any viewpoint as insane as normalizing deaths during the development process is totally incompatible with successfully deploying a safety critical product. Deaths may be unavoidable due to the sheer weight of statistics, but deaths are always a failure of immense gravity. A flippant disregard for human life has no place on a safety critical product.

> > Not everything, just things that can kill hundreds to thousands of people if they go wrong

The original Al Quaeda plan for the WTC was to detonate explosives to tear it down, they tried in the 90s but of course they were so inept that they just made a huge mess in the parking lot.

However let's say they succeded. Do you think we'd have checkpoints looking for explosives (or even components to make explosives) while entering Manhattan and all the 1000 odd major global cities which exceed 1M population?

There is a thing with flight. Planes look so big and so gross and like they should not be able to fly and so people are scared of them, fear of flight is one of the most common fear.

In the post 9/11 world add to that being under a plane, in the proximity of a plane or in a builing from which a plane can be seen.

None of this stuff happens with trains and they too can kill thousands.

I am not really sure what your point is. I think the argument thread is:

1. Root claims aviation processes are a good example of safety processes that self driving cars and other safety-critical industries can adopt.

2. You disagree claiming that aviation processes are too onerous citing TSA and ground security processes. In addition, you state “If everything were to be as safe as aviation the whole world would come to a screeching halt standstill.” You are implying that the only choice is between aviation processes everywhere or nowhere.

3. I pointed out that high safety processes only need to be applied where it matters. You can apply challenging processes on a case by case basis depending on the risk.

4. I guess you are now stating that aviation processes are excessive over reactions because in no other industry would they do the same things? And thus they are a poor model that should not be adopted by other industries since aviation is too risk averse?

To which I say, okay? Aviation is not the only model of safe processes, it is just a model with a very good safety record, probably the best. It is useful for establishing what an upper bound on safety processes might look like and cost. But, as you point out, the cost/benefit may not make sense for other industries.

The point is that there is a vast range of processes between nothing and aviation. Something closer to aviation or trains rather than total recklessness is probably more fit for safety critical systems that can kill hundreds to thousands of people.

> > The point is that there is a vast range of processes between nothing and aviation. Something closer to aviation or trains rather than total recklessness is probably more fit for safety critical systems that can kill hundreds to thousands of people.

To me the point is that there are many things that are used by billions of people and can kill thousands of people per incident when things go wrong, if even one such system has zero safety (for whatever reason ranging from economics to culture) then it makes sense to roll down safety of every other such system to zero as well to pursue quality of life between now and the not so distant moment when something goes wrong

So if you safety processes made 99% of situations 1,000,000 times safer, but 1% of situations remain high risk you may as well throw out all your processes and incur 1,000,000 times more risk 100% of the time?

That is positively absurd.

It might make sense to reevaluate what is the most efficient allocation of resources which would result in the greatest holistic safety increase per resource spent, but throwing everything out to reach the lowest common denominator is just straight up nonsense.

The only way that makes sense is if you assume a completely uniform risk tolerance, completely uniform mitigation ROI, and you assume the highest risk activity is the true underlying uniform risk tolerance with the other 99% of low risk activities being the outliers. Literally every one of those assumptions is ludicrous.

You are making up stuff that is not real world, the reality is that billions of people take a plane every year, but the same billions of people also roam into malls, stadiums, parks, buildings and generally areas that are completely vulnerable and unprotected against ill intentioned individuals.

It's irrational to use this level of paranoid aviation safety because the same people also do all kinds of stuff that isn't remotely as safe against ill intentioned individuals

The commenter you're responding to is talking about aeronautical engineering safety practices (things like DO-178), not TSA-style security. If you haven't worked in the aerospace sector, it's hard to appreciate just how slow, methodical, and bureaucratic things are.

You say we need to investigate autonomous vehicle crashes, I agree.

We haven't done this with people because there's not much to investigate; someone drinks, drives, and crashes, there's not much to gain from serious scientific investigation here. Autonomous vehicles are different and we would benefit from more focused scientific investigation of autonomous vehicle accidents.

The idea that there's not much to investigate after an accident — that every accident is idiosyncratic — is one of the great lies of the auto industry (and their buddies, transportation planners).

> We need to hold companies accountable for fixing flaws in autonomous vehicles.

Don't we even just start getting a chance at doing that with autonomous vehicles? When a car crash happens and driver is found at fault, we can say "let's fix driver training to prevent this from happening again!" but humans are diverse, many don't pay attention in driver's ed, and they retain learnings at different rates, so this isn't going to work very well. With a self driving car, at least, since the software is replicated in many vehicles (while humans aren't), we have an actual chance improving on the situation over time.

Reply on news.ycombinator.com