Hacker News Reader

Comment by cashsterling | Hacker News Reader

Stuff like this is completely expected of nonlinear, high dimensional controllers (and neural networks used for object detection and decision anlaysis fit into this category)...

NN's are not fit for high-predictability, high-safety-factor control. I wonder if, in fact, we cannot construct a NN controller with the level of unusual object detection and edge-case scenario recognition required to match an average human drivers capability to recognize and react appropriately to unusual situations. I mean to say that I think this is something like an NP-hard problem... to make it one-log better you need 100x the data and 100x the NN dimensionality. Model complexity explodes...

I also think we are spending a lot of time and effort solving the wrong part of the transportation problem. Cars have a thermodynamic problem for society --> a 1500kg car to move 1-2 80kg people is very energy inefficient and resource intensive to manufacturer.

We need remote work, more home delivery options (e-truck/vehicles), greater incentive for electric bikes, and better/more public transportation.

my 0.02...

View on HN

Cars should not detect "obstacles". They should detect roads. They should not seek _exceptions_ to driveable space through classification- or segmentation-first strategies. They should only detect driveable space. Full stop.

You either detect a full-stopping-distance's worth of clear road through reliable (non visual) ranging with dense enough samples to preclude wheel-catching voids ... or you will continue to run into "objects" which slipped through your classifier.

I can accept reduction in this margin (e.g. in dense traffic flow) when it is first shown to work with margin.

Is this a random thought or something that has seen more debate? Are there articles or papers comparing this approach to the other?

The problem that almost all current AI research has is that nobody knows if your model is detecting the obstacle or the absence of a road.

The cliché example is when people tried to build a tank classifier. Since they took the pictures of the different tank types on different days, the AI learned to detect the weather conditions instead.

So to determine if your AI is detecting the roads absence, you would first need a lot of testing images that look realistic in general, but with absent road. As far as I know, such a dataset does not publicly exist yet.

So yes, that commenters approach seems new to me.

Do you have a link to that tank/weather study? I've never heard of it, sounds really interesting!

It may or may not have happened with tanks; it sure happened with horses:

To understand how their AI reached decisions, Müller and his team developed an inspection program known as Layerwise Relevance Propagation, or LRP. It can take an AI’s decision and work backwards through the program’s neural network to reveal how a decision was made.

In a simple test, Müller’s team used LRP to work out how two top-performing AIs recognised horses in a vast library of images used by computer vision scientists. While one AI focused rightly on the animal’s features, the other based its decision wholly on a bunch of pixels at the bottom left corner of each horse image. The pixels turned out to contain a copyright tag for the horse pictures. The AI worked perfectly for entirely spurious reasons. “This is why opening the black box is important,” says Müller. “We have to make sure we get the right answers for the right reasons.”

https://www.theguardian.com/science/2017/nov/05/computer-say...

There is, in general, a great deal of work on explaining the decisions of neural net. Explainable AI is a thing, with much funding and research activity and there's books and papers etc, e.g. https://link.springer.com/book/10.1007/978-3-030-28954-6.

And all this is becaue, quite regardless of whether that tank story is real or not, figuring out what a neural network has actually learned is very, very difficult.

One might even say that it is completely, er, irrelevant, whether the tank story really happened or not, because it certainly captures the reality of working with neural networks very precisely.

And as a less dataset-quirk-specific example of AI fixating on backgrounds, here's Microsoft Azure dreaming of electric sheep: https://aiweirdness.com/post/171451900302/do-neural-nets-dre...

What wonderful examples, thank you for sharing :)

Nice, thanks :)

Incidentally, (human) kids should never be allowed to hug sheep or goats like that. They can easily catch something nasty (enterotoxic E. coli, mostly). See e.g.:

https://www.bbc.co.uk/news/uk-england-lancashire-35039878

Juliette Martin, of Clitheroe, took her daughter Annabelle, 7, to the 'Lambing Live' event at Easter last year.

The youngster, who had bottle-fed a lamb, suffered kidney failure and needed three operations, three blood transfusions and 11 days of dialysis.

It doesn't seem to have happened. Gwern has done some extensive research: https://www.gwern.net/Tanks

As I say above, even if the tank story is apocryphal, it captures the tendency of neural nets (modern or ancient, doesn't matter) to overfit to irrelevant details (which, btw, is what Layerwise Relevance Propagation from my comment above is trying to determine).

This is probably the reason why this story has been repeated so many times (and with so many variations): beause it rings true to anyone who has ever trained a neural net, or interacted with a neural net for any significant amount of time. Unfortunately, the article you cite chooses to suggest otherwise.

In any case, if the tank story is an urban legend it has its roots firmly in reality.

I understand that the scenario rings true, but Plyphon_ specifically asked for "a link to that tank/weather study", so the fact that it doesn't seem to exist is of primary importance.

From the tone of Plyphon_'s comment it seems to me pretty obvious that they had at least read the article you link above and the point of the comment was not to actually get a source for OP's comment, but to score some kind of internet burn points. Judging from the earlier greying-out of Plyphon_'s comment I'm not the only person who thought so. I don't think that kind of comment should be validated with an actual response. It is borderline snark and certainly does not contribute anything to the conversation.

I'm happy to accept my mistake if I have misunderstood Plyphon_'s comment.

Computer can draw what it sees at windshield, for driver, by tracing position of driver head, to position projected image correctly. E.g. draw red outline around obstacles, draw green outline for clear road, draw predicted trajectories in orange, draw selected path in blue. Human will see what computers sees, so it will be able to react quickly.

I think this is the right approach.

However one big problem with deploying systems using this approach today is that it happens to be very hard to do this with enough nines that your car isn't slowing down or emergency braking for spurious inputs. Floating plastic bag / leaf - trash on the road - asphalt line that looks wrong - camera glare - a puddle with a reflection - giant sticker of a photo of a traffic cone - etc. etc. etc. State of the art would probably be undriveably cautious today.

Most things that don't look like road, between lane markers on a highway, are still drivable space. Probably 99%.

That doesn't mean it's the right approach. Just that the alternative isn't really deployable today.

I agree. To elaborate, I think that "Looks" is the problem. Relying on appearance is a fundamental mistake. A road is a predictable flat surface that is safe for wheels at speed. If the map matches your ranging probes, and no geometry that is non-trivial sized impedes or will impeded your progress, you can drive.

This should immediately preclude any approach to self driving cars that has pedestrians, bags of trash, debris, cross traffic, etc. Highway (and probably daytime) only. I don't understand the obsession with starting with corner cases and working back to the very real good that self-driving autonomy can do on highways at speed. Build back with lessons learned, rather than solving the whole problem in one shot. That seems like fundamental engineering, but the marketing team is in charge, not the engineers.

Until I started peeking into how self-driving cars worked, I understood that this was the standard way to build a self-driving vehicle. It still is, outside the marketing-driven industry.

Still there are some exceptions. Kid has collapsed on a lane and lying down on the lane. In a way that line markings are NOT crossed and head is further than legs from Car's (and your) point of view. Car MUST stop.

Or maybe that's just a fox that's lying down there - and depending on road, speed, etc - you take more risk dodging, breaking and car SHOULD just go.

Just one example I immediately thought of.

Any anomaly makes the space non-driveable. It’s a road. An autopilot should not be entrusted to decide further, beyond anomaly detection sufficiently far ahead to allow for braking.

But of road becomes non "drivable" at a point ahead where you can't stop in time you need to evaluate what to do (hit it, try evade, etc.) which are all hard decision.

> An autopilot should not be entrusted to decide further, beyond anomaly detection sufficiently far ahead to allow for braking.

This isn't possible. Not just technical but theoretical. Situations in streets can change way to fast. Both through external factors and other cars. Even on a straight highway with fences on the side to prevent anything from going on it this would still but work. Once it happens an intermediate decision is needed which can not be reached on time by delegation to the driver (as the driver want driving you can't except a good reaction speed even if that person had the eyes on the road).

Eliminate situations where you can’t stop in time. If impossible, then autopilot should not be marketed as such.

On highways, enough clearance from the sides of the road and wide enough detection beam cone may be able to help. Yes, I imagine in such a case high-speed motorways would warrant security measures similar to those around high-speed train tracks.

On denser city streets where restricting pedestrian access is not feasible nor desirable[0], appropriate (yes, probably pretty low) speed limits could reduce braking distance and hence the chance of a tragic accident to below what currently happens with human drivers.

[0] This concerns existing streets. For new areas, infrastructure that separates car and pedestrian traffic with tunnels/bridges and accessible over-/underpasses could address that naturally.

Love this idea. If autopilot cannot be engaged on an unsafe road, then maybe we will actually complain enough to improve roads and supporting infrastructure

> This isn't possible. Not just technical but theoretical. Situations in streets can change way to fast.

Then drive slower. It's a speed limit not a speed minimum.

When I was training for my drivers license (in Denmark) my instructor told me to never drive so fast that I wouldn't be able to break to a complete stop within the current visible distance.

My instructor (in the UK) gave me the same instruction. To be fair, it's pretty intuitive. It's also remarkable how few people follow it (e.g. they don't slow down sufficiently around blind bends, or when there are pedestrians close by).

If there were anywhere that could have had full autonomy even a decade ago it would be train networks, and it's not for lack of obstacle detection technology that they don't. The problem is contextually dependent enough even on an entirely linear rail network that humans need to add their problem solving finesse regularly.

I just don't think autonomous cars is a problem worth working on if the requirement is that they need to work on existing road infrastructure. We are going to get half baked, not-really-autonomous vehicles and the manufacturers are going to shift blame to the drivers because they should have known better than to use their vehicle's advertised autonomy in (x) specific circumstances, where x is the infinitely variable realities of the outside world.

The idea originally expressed by jvanderbot presumes dense non-visual (e.g. LIDAR) scanning and simple anomaly detection without AI “black boxes”.

With that in mind, is this really a question of context? The presence of an entity that reflects LIDAR should be enough of a reason for the system to brake.

Yes, this would imply lower threshold of what would be a situation that warrants braking, and hence on many ordinary streets autopilot may become unusable or car movement could be perceptually too slow.

To your point about rail network, autonomous rail lines do exist—I’ve used the new light rail system in Macau, and trains are entirely driverless. (From my observation of their behavior, they do not appear to be controlled remotely either.) I imagine some factors that impede innovation in this space do apply to cars (e.g., track security), but others don’t (guaranteeing stopping distance for long trains with heavy cargo, keeping people employed, arguably less flexible infrastructure).

Of course it's still a question of context, there are more factors are play than "Should I stop, Y/N", such as "Will stopping actually work given the conditions, my payload, is the object moving toward me. Should I swerve instead?"

My point is if the best we can do is a LIDAR detection auto-stop then that's not full autonomy, and I question how close we'll actually get to contextually aware autonomy.

> To your point about rail network, there are autonomous rail lines—I’ve used the new light rail system in Macau, and trains are entirely driverless.

That is actually really cool to hear. I know I come across as a Luddite by having a pessimistic view on car autonomy, but I will be happy if we can crack it.

> Will stopping actually work given the conditions, my payload

If there is rain and/or your payload is heavy, that increases your braking distance, so accordingly detection should either work further ahead or your speed would have to be reduced to allow for safe braking.

> is the object moving toward me

This is a very valid point. If not all vehicles are autonomous, even on a well-protected highway it is possible to have a cascading accident caused by human mistake. This would even be an issue if all vehicles are autonomous, since some could be hijacked by owners.

In my view this type of “dumb” object detection for emergency braking purposes should work together with contextually aware AI-based overall autonomy, not instead of it. Upthread I wrote “an autopilot should not be entrusted to decide further”, which was perhaps ambiguous—meant that in context of an anomaly ahead.

Without some incredible breakthrough in AI, trying to adapt autonomous cars to the regular road network is like swimming up a cascade. We're far more qualified to adapt the road infrastructure to autonomous driving, perhaps even remove most of the cars and driving in the process, leaving only the bare minimum.

Ok, a wind blows a empty paper bag on the road. A human will see by the way it moved that it is empty and safe to drive over. What will AI see? A driveable road or a road with a rock on it? I still think that without the "model of the world" no AI will be capable of driving. And that model requires GAI.

I'll take a self-driving car that makes a controlled stop for a wind-blown paper bag over a self-driving car that ploughs into an overturned truck.

I agree but stopping on the highway for a paper bag could itself lead to serious series of accidents

It's because human beings are unable to keep safe distance.

Scapegoating of humans who make soft errors doesn't do anything to make autonomous driving safer.

I think the vehicle should alert the driver, and being controlled slowdown until it is overridden. Sorry, driver, you are inconvenienced. Maybe you are in the carpool / AI-only lane, so it's not a big deal. Yes, there probably should be an AI-only lane for now.

This could exist now, but we have set sights on Level 5 at the expense of pragmatism.

Interestingly, I remember this exact scenario was a noted problem for some sort of computer-controlled shock absorbers that I saw on the "Beyond 2000" TV show in the mid-90s. In many scenarios, the shock absorbers could detect bumps and holes in the road and compensate perfectly, resulting in a ride that was smooth as glass.

But they still hadn't figured out how to differentiate between a crumpled paper bag and a rock, or a pothole filled with water vs a smooth surface. Otherwise, it was really impressive.

Found it on YouTube: https://www.youtube.com/watch?v=3KPYIaks1UY

I feel like Level 4 or 5 Autonomous Driving will be the same -- there are just some problems that will never be solved well enough to make it work acceptably.

Not that easy, let's say there is a car coming from the side. Will it cross your own trajectory or not?

That's actually some easy math, isn't it?

Well, I think that you can't predict if the driver of the other car will brake (or accelerate).

And even if: are its breaks strong enough to prevent it from crossing your way? (Or the other way round: What is likely its maximum acceleration?)

Please note that all of the above can happen even without any adversarial intentions.

The difficulty of the situation in general is why limitations should be placed on the system and a conservative approach taken in practice. Trying to circumvent difficulty with complexity is not an option for the first roll-out of a product.

Deploy self-driving on marked single-lane AI-only highways. Maybe let self-driving cars use bus lanes. But that's it for now, until we can certify and understand limitations of a system. It's only fair to the public to have slow rollouts that require mostly-attentive driving.

So, turn roads into closed train tracks? That will take even longer than creating a true AI... people drive cars, people cross roads, stuff gets on roads, there are so many vehicles some do break down... What you suggest would lead to self-driving cars turning most roads into endless traffic jams.

That sounds like speculation. But anyway, there's no reason to open the floodgates in an unprincipled way just because being principled and iterative is challenging.

This doesn't work because laterally moving obstacles can enter the roadway within your stopping distance buffer. A self-driving car MUST be able to identify obstacles, especially moving obstacles. Your stopping distance at highway speed may be equivalent to one football field, but you may be able to anticipate obstacles if you identify laterally moving objects before they enter the roadway.

Your suggested scheme for driving offers zero protection from cross traffic cutting in front of the autonomous vehicle, whether at a stoplight or otherwise.

Moreover, not all non-driveable space is created equal, so if the vehicle has to swerve to avoid a collision, it should be able to identify the difference between, say, a clearing, a pedestrian, and a pond.

These are totally basic principles with which any driver (and especially any motorcyclist) would have no trouble. The problem with your imperative statement is that it produces a pointlessly elegant, practically bad solution.

"First" does not mean "exclusively". But detection of an impending obstacle should not start with classifying that obstacle. You need only know that some geometry that looks larger than trivial is moving at a vector that will impeded your stopping distance requirement. Until that is proven to work, it is pointless to try to "understand" the object or its intent.

I'll go further. To avoid clearings, ponds, and pedestrians, Self-driving cars should be highway-daytime only until proven reliable. Starting with the edge cases and building back to the easy ones is ridiculous engineering policy.

Peripheral vision and detection is extremely important to avoiding accidents... what about cross traffic or children running into the road (as others have mentioned). Detecting traffic lights that are out... etc.

If you turn up the forward looking anomaly detection and safety factor too much... well, among other things, the non-linear controller is going to emergency brake for no good reason (Tesla owners have already reported this). Spurious emergency braking is extremely dangerous for traffic behind a vehicle and the danger increases at higher speeds where the controller will be even more touchy.

Methods for predicting "drivability" or "clearness" of roads based on moving object detection are not different than what I said and are not novel in R&D / literature or practice. I'm asking for exactly that, absolute conservatism first.

In my opinion, autonomous driving should be highway only, daytime only until proven effective in mass produced vehicles. The hype about rushing to level 5 is marketing, not a feasible technology roadmap.

> a 1500kg car to move 1-2 80kg people is very energy inefficient and resource intensive to manufacturer.

Cars used to weigh /less/. The weight is the price of all of our drastically improved safety systems. Either way, you're stuck with a lot of that infrastructure if you want to move food and other goods between cities. All of that is the price of flexibility in our logistics.

The great wonder of our time is that we can do all of that _and_ start the long process of deprecating using fossil fuels. Now, if we can build the cars to last much longer than we currently do, we're really starting to win.

> and better/more public transportation.

I feel like there's a hidden trade off here as well. I think the "better" and "more available" you make public transportation, the less dense your ridership gets on certain lines, and the less likely you are to truly gain any efficiency across the system.

> less likely you are to truly gain any efficiency across the system

Urban sprawl is part of the problem and should be addressed at the same time.

I think that your definition of "NP-Hard" is incorrect. It would be better to instead say "Exponentially bound in required training data size"

I say this because having a solid model of these terms is required when making "intuitive conjecture" about what is and isn't feasibly possible. Driving is NP-Hard? It would hint that humans have brains capable of computation more powerful than a turing machine. Which... suffice it to say we probably don't. I'm not aware of any proven algorithms that humans can compute which computers cannot.

In images free of distortion and noise, computers were outperforming humans in image recognition 2 years ago! [0]. Since then, with advances in GANs (which specifically addresses the noise/distortion issue), I suspect we are close to achieving super-human image recognition. The last missing piece is "context" or prior knowledge. If you only see a whisp of hair sticking out from behind an object, you have prior knowledge that hair grows on humans, and that there's probably a person there. This last piece is being addressed by multi-modal networks [1].

If you have any doubt of the power of computer vision, my last hope is to link you to this paper from 2017, look at the "regions-hierarchical" results:

https://cs.stanford.edu/people/ranjaykrishna/im2p/index.html

[0] https://arxiv.org/pdf/1706.06969.pdf [1] https://www.themtank.org/multi-modal-methods

The mistake is believing this is a training data problem. You may have never seen an overturned truck before, but if you spot one on the road in front of you you'll easily figure out you need to stop. You don't even need to know exactly what it is, you just need to know A) you are going to do damage to it or vice versa if you hit it, and B) it's not getting out of your way.

-- Cars have a thermodynamic problem for society --> a 1500kg car to move 1-2 80kg people is very energy inefficient and resource intensive to manufacturer.

I think there's a good gap in the market to be made by E-Bikes, but it needs regulatory help (Bike lans with barricades) to make people feel safe enough to commute.

My reasonably old and frail father has an E-Bike and he zips around town on it. The hill assist is great (Though i did enjoy the slightest bit of schadenfreude when he's battery died before the return uphill to his house on day) But he lives in a town with a lot of bike paths, so he's not really on the road.

I think if you can break open that market you could make driving safer and make biking safer. It's just right now driving is a wealthy gabmit. Rich people drive tesla's and value signal with them it's not a new issue but getting people onto bicycles would be great for everyone on the road.

Yeah except this crash was on the highway. No one is taking ebikes on the highway. It’s arguably the most controlled environment there is for cars outside of a test track.

But once autopilot works on the highway you don't think they aren't going to try and corner the market with inner city driver assist.

Regular use of (e)bikes within a city can make people sell their cars, and use other means of transport for occasional longer rides. Since I replaced my car with a bike, I also spend much less time in a car on the highways.

Cars don’t really require all that much energy. A 15,000 mile per year car at 4 kWh per mile is 10.3 kWh per day. You can get that much solar power for the fraction of the price of the average new car.

Roads, human lives, traffic congestion, etc are much larger issues than energy. Over 20% of NYC is devoted to cars and they don’t even come close to covering the cities transportation needs.

But maybe 50 kWh a day is embodied in the manufacturing/depreciation of the car, I'd estimate.

76000kwh at 50kwh/day is about 1500 days, or 4+ years.

https://blogs.nicholas.duke.edu/citizenscientist/embodied-en...

The average car is a smidge under 12 years old, which is about 17kWh/day.

I was assuming 15 years, but I was also assuming too high an energy intensity; please see my other follow-up.

Where does your estimate of 50 kWh per day come from? According to [1], Tesla's 2017 manufacturing and delivery released 186000 metric tons of CO2, most of that "indirect" (as in at their suppliers). (146000 for "Facilities" and 39000 for Sales, Service, and Delivery.) According to [2], they delivered 101312 vehicles in the same period. That is 1.8 metric tons of CO2 per delivered car. Your rate of conversion between kWh and metric ton of CO2 may vary, but the US average is about .99 lbs/kWh [3] or 3640 kWh per car. If we assume a car lasts 10 years, that is about 1 kWh/day. Also these numbers are from back when the made almost entirely Model S and X, so it is probably with a 3-filled fleet.

[1] https://www.tesla.com/ns_videos/tesla-impact-report-2019.pdf , page 16

[2] https://ir.tesla.com/news-releases/news-release-details/tesl...

[3] https://www.eia.gov/tools/faqs/faq.php?id=74&t=11

I was assuming a car that cost $30K and lasted 15 years.

The parent seemed to be talking about cars in general, so I wasn't referring to Teslas in particular.

I can try this again using some different sources and methods.

- GDP of $1000 equates to about 1540 pounds CO2.[1]

- Each pound of CO2 implies about a kWh based on average power generation intensity.[2]

- Therefore, a $30K car implies 30 x 1540 or about 46,200 kWh.

- Divided by 15 years and by 365 days gives about 8.5 kWh per day.

So, I was probably off by a lot, and may still be off, but I feel like my point is intact, because 8.5 kWh still almost doubles the energy requirement originally stated. It's not negligible.

[1]https://en.wikipedia.org/wiki/Emission_intensity

[2]https://www.eia.gov/tools/faqs/faq.php?id=74&t=11

I see. For a regular gas car, the 4 miles per kWh is wildly optimistic -- 134 miles per gallon. The kWh per mile (just from tank to wheels) would be something like 4 times higher than that. (1 kWh per mile = 33 miles per gallon). So that 8.5 kWh per day from manufacturing would be on top of something like 41 kWh per day of driving, not the 10.3 from the post up there.

Sure, I get kind of lost trying to figure out what you are saying when you're addressing me about someone else's comments that neither of us may fully understand the thinking behind.

However you look at it, Tesla or not, 10 kWh leaves out a lot which was my original point.

Cars last far longer than 15 years on average. For the average car in the US to be almost 12 years old means they must last on average about 25 years as the rate of car production has been increasing.

Secondly, GDP to CO2 figures are a poor fit. Agriculture for example is directly 10% of US CO2 emissions but only 5.4% of GDP.

I think 15 years is a good estimate.

First thing that comes up when I Google:

"While an average lifespan of vehicles is not given in the IHS report, a 2014 Automotive News article stated that, at that time, the peak lifespan or “scrapping age” of a vehicle was 13 to 17 years old"

Scrapping age relates to when the car is no longer worth repairing after an accident. In the US many cars also get exported when their not worth keeping in used car lots. Still a huge number of older cars are on the road and many of them are far older than 25 years.

From 2014: https://www.nbcnews.com/business/autos/old-clunkers-rule-roa...

How many of these "pre-'99" models are still around? 53.3 million.

4 miles per kWh, I think you mean.

Ops, yes. To be clear, 15,000/4/365 = 10.3 kWh per day is still correct.

10kWh per day is also only one third the average daily power consumption of an American household.

Your estimate is 4 miles per kWh, or about 135 mpg. But the average car in the US is about 0.75 miles per kWh, or about 25 mpg

4 miles per kWh is what Tesla's Model 3 does (and similar EVs).

ICE cars lose a lot of the energy stored in gasoline to waste heat, especially in city driving.

That's to say nothing of the challenge of generating gasoline from solar energy.

Even my Volvo XC60 PHEV(2.2 tonne SUV which also+ carries around a petrol motor and a tank of fuel) averages about 2.5-3miles/kWh in pure electric mode. These numbers can only be improved going forward.

Bear case for self-driving: in 1 million miles of driving, roughly the number between major human accidents, do we really think these cars will not encounter one AI-complete problem.

In order for there to be a major accident, there not only has to be a mistake, it has to be a mistake with a major consequence, and it has to be that nobody else there avoided it either.

In general a car that doesn't see a pedestrian isn't a major accident unless the pedestrian also doesn't see the car. If not for that there would be a whole lot more fatalities with human drivers as well. A self-driving car obviously shouldn't do that, but neither should a human, and they do. Which makes it less hard for the computer to do it at least as well.

You can also expect the computers to improve over time by learning from each others' mistakes, which human drivers generally don't.

While it is the case that it usually takes two people to cause an accident, there are plenty of scenarios where there is only one person who is capable of taking the action necessary to avoid it. For example, an elderly person in the crosswalk is not going to be capable of getting out of the way of a speeding car.

It's already the case that pedestrian/car collisions are usually written off as the pedestrian's fault until proven otherwise--the Uber self-driving car that killed a pedestrian is a great example of that in action.

And the same is true of human drivers, is my point. You don't just need the mistake, you need all the other things to go wrong too. The pedestrian entered the crosswalk even though a car was coming, instead of waiting to make sure the car stops first (as pedestrians often do). The car failed to identify something. The thing it failed to identify was a pedestrian and not a mailbox or a trashcan. The pedestrian was elderly and couldn't jump out of the way fast enough etc.

It all has to go wrong at once. Mistakes are common but most mistakes aren't fatal.

> Mistakes are common but most mistakes aren't fatal.

And by choosing to drive a car, you are dramatically raising the odds that any mistake you make is fatal (not necessarily to you).

While my personal experience on the matter is thankfully limited, I strongly suspect that the vast majority of vehicular-pedestrian accidents are mostly (or entirely) the fault of the driver, where the pedestrian started an action where it was safe to do so and the driver rapidly caused it to become unsafe before the pedestrian could react.

Is 1 million miles really the approximate distance between major accidents? Because in a lifetime, many people would approach that, and yet most people don't die in auto accidents.

According to AAA, in the US there is approximately one injury accident per million miles driven, and one fatal accident per 100 million miles:

https://aaafoundation.org/rates-motor-vehicle-crashes-injuri...

I think most people are in a major car accident at some point in their life. Major doesn't mean death.

Besides what the other replies have said, you can also work backwards from insurance rates and margins, and it generally tracks.

It is perfectly reasonable to expect these cars to do at least as well as humans at avoiding mind-bogglingly stupid scenarios that the least experienced driver could avoid while drunk, talking on the phone, and applying makeup. This is absolutely one of those scenarios.

If there was an autopilot that nailed every scenario in any weather condition except clearly overturned trucks in good weather, should that system be approved for general public use?

Considering the fact that accidents often attract bystanders and emergency personnel to help the incapacitated driver, the inability to avoid an existing wreck in clear weather is a glaring hazard to human life.

Moreover, trucks sometimes contain hazardous or flammable materials.

So the answer to your question is a resounding no, in my view.

Huh, on the balance of things refusing to use such a system would cost a lot of lives.

Only if there weren't alternative systems that were just as good and also managed to avoid this problem. Which seems unlikely.

> Which seems unlikely.

No kidding.

https://www.cdc.gov/niosh/docs/2001-128/pdfs/2001-128.pdf?id...: The two most dangerous things in road work, accounting for half of all of the industry's fatalities, are industrial vehicles working on the site and passenger vehicles passing it.

So, yeah, you're right - there aren't any alternative systems that manage to avoid this problem. Even humans suck at it. And if even humans can't handle construction areas safely, how can you possibly expect computers to be perfect?

* It depends on why the humans are failing. You can design a computer to not speed and not get distracted.

* Hitting a worker is very different from hitting a truck.

* Many of those fatalities are not in clear daylight.

There are. Use two or three alternative AI implementations, which will watch each other. Kind of famous "Predator" algo, but for driving. If one will fail, second will pick up. If one makes mistakes, second will teach it.

You can bring in new safety features without having full self-driving. I think the safest option is to have human drivers, with the car always monitoring to avoid any serious accidents. People will still be attentive, since they are in full control, but you avoid just as many accidents as you would if the car was driving itself.

I agree. By establishing an "auto-pilot," you actually eliminate redundancy by putting the driver in a passive position. (No matter how many times you tell the driver to remain attentive)

The machine ought to be an error handler for specific failures of an active human driver (sudden braking of lead vehicle, lane straying, etc). This is the only way to get both machine and human to pay full attention. A person in a co-pilot role will struggle to react quickly enough to handle errors of the machine auto-pilot.

Yeah. This is what I always thought. If we could safe many lives with self-driving cars then we could safe many more with augmented humans.

No it should not.

Why not? In that hypothetical, ramming every overturned truck while avoiding all other accidents would be a massive improvement in safety. What about the truck scenario makes it so bad as to outweigh avoiding the other accidents?

It demonstrates that the "AI" can't avoid basic obstacles that a 16 year old kid driving for their first time could avoid, of course.

From a political perspective, almost certainly not even if it would ultimately save lives

Nope, it shouldn't. A system that happily plows into a massive static object that blocks 3 lanes of traffic just because the system didn't recognize it as a danger should never be allowed on the road. In fact Tesla should remotely disable autopilot on every single car out there until this is investigated, tested and patched. And even then I have my reservations about the naming, it's not an autopilot.

The other scenario is whether fully autonomous vehicles out-perform humans in sufficiently many other scenarios such that we will consider the comparatively rare scenario of an overturned truck not that big of a deal compared to the massive numbers killed due to raw human carelessness. Furthermore, autonomous vehicles will likely get better and better at driving collectively, something which I would make a heavy bet against for human drivers.

No. For two reasons:

1. Technology should not make mind-bogglingly stupid mistakes like this, regardless of how rare they are in the grand scheme of things.

2. Because these systems use neural nets, they're black boxes and it's impossible to conduct a proper engineering analysis on them to find out when they will fail. Fuzzing the system with live humans is not engineering; it's callous disregard for human life and anybody who practices it should lose the right to call themselves an engineer.

We hold machines to much higher standards. We accept human imperfection and human error. People killing people on the road because they are sleepy or not paying attention is a risk we are willing to accept. The reason is that people have a skin in the game themselves. Being in a accident means they risk injury, fines, jail time.

I think few are willing to accept autopilots that are only as bad as (or as good as) humans. They must be significantly better than human drivers, simply because we don't accept their errors like we do human errors.

This is completely irrational when looked at purely from a safety point of view (Who doesn't want safer roads?), but this obstacle is very real.

Worse: that autonomous cars are better in many scenarios or aspects (they don't get sleepy, they are never drunk) doesn't mean we accept that they are ever worse in any other aspect, despite the total safety being better.

That is: autonomous vehicles must not just be as good as human drivers, they must be significantly better (or at least safer) than human drivers. And not only that, they must be significatly better in every aspect of driving, for them to gain acceptance.

It is not irrational because self driving cars are not interchangeable with human drivers 1-1. If autonomous cars are put on roads it is very likely net traffic increases. We put up with bad driving because we have to, this is absolutely not the case for machines. I, personally, don't want more average drivers on the roads.

The problem is that the flaws of humans tend to be random and stochastic, while the flaws of machines tend to be consistent and systematic. This is the at least the fifth example of Tesla missing stationary objects on the road.

Humans might have a judgment failure every so often, but they might realize how unsafe they are and change their behavior, or drive less, or get their license taken away. A machine consistently crashing cars has no self correction protocol, and will consistently make the same mistake over and over.

> greater incentive for electric bikes

Greater incentive for any sort of bike or small/low-tech personal vehicle, imo

I agree with the idea that car-based transportation is inefficient. However, people will not get rid of it until they are forced to.

People don't have to be forced to get rid of cars, they just have to be incentivized to get rid of cars. That can be done by increasing the convenience of other options compared to cars or by decreasing the price of other options compared to cars.

It would go a long way to even make it possible to get rid of cars. There are huge swathes of America which aren't within a mile of mass transit. And you can't fix that with more mass transit, because the density there isn't high enough to justify it -- a bus with one rider is worse than a car.

What you need is to get rid of zoning density restrictions. That's the only way for more people to live close enough to things for walking and mass transit to be viable.

> There are huge swathes of America which aren't within a mile of mass transit.

> What you need is to get rid of zoning density restrictions.

Most of those places were built for and are still dominated by the votes of people who don't want more density and transit options, for a variety of reasons, some reasonable, and others based in the nation's sordid racial history.

I don't agree with those people, but until the settled, voting population inhabiting those vast swathes gets significantly younger and more pro-transit and density, I don't see what you call for happening soon.

This is one of the few instances where doing something at a more central (e.g. statewide) level of government could be productive.

That's exactly what the state of CA has been doing with recent state bills that override local density restrictions.

The problem with the US is that people are scared of buses. A bunch of people from SF told me the buses were not good, but when I visited I took several and they were consistently good, and were never empty, just the frequency could be a lot better. They were a lot noisier than the buses in Europe, though.

You are greatly minimizing the problems that most Americans face. Most American cities are not dense enough to require good public transportation options (the exception being NYC); without enough density (and lack of public funding), public transportation is short on cash and often very inconvenient. If you miss your bus, you have to wait an absurd amount of time for the next available one. Also good luck with hauling in all the groceries and stuff you need into your suburban home using only buses.... simply not possible.

So there are multiple factors at play. Short of a massive change in the way American cities are designed or some innovation that greatly reduces the cost of building public transportation, cars will continue to be something Americans absolutely need for surviving.

My understanding as an average American (in this context) is that in European cities there are nice, fairly frequent, commuter trains and rail; and often also busses.

In the US, mostly the only 'good' bus service is geared for 9-5 workers in big offices in big cities. Anything outside of that tends to run across the economic divide that: people who take the bus _have_ to take the bus, while taking the car is a luxury in time, freedom, and isolation from possibly literally unwashed masses (who might also literally be taking the bus in the summer for a place to have some AC and safety with a seat).

The issue is very much related to classism and a lack of social safety nets and inclusion.

Another factor to consider is the age of the cities involved. Most European cities predate cars, while many American cities were built up after or during the the rise of the automobile.

"Most European cities predate cars"

I suspect that only applies to the cores of European cities - most have got much larger since the car was invented and the newer areas were definitely designed with cars in mind.

e.g. Maps of Edinburgh over a few centuries:

https://maps.nls.uk/towns/edinburgh-city.html

Even in the 1920s Edinburgh was pretty small compared to its current extent.

You say classism like it was extrinsic. But the classes (in smaller cities, not like NYC) are people who can afford a car and people who can't, or have some other reason they can't drive.

In the sort of place I'm used to, an urban area of about a million people, taking the bus is not logical if you have a car anyway. You don't get an economic benefit if you own and insure a car at all. And you pay a big penalty in time and flexibility.

That's what creates the class divide, not something arbitrary in peoples' heads. The people who ride the bus because it's economically rational are inherently the people who have no choice.

Talking about general social problems as the source of the issue is misguided, in my opinion.

You're conflating being able to afford a car with having a car. Ideally more people would choose not to have cars. Or, easier, a household could only have one car instead of multiple.

And a good transit system can beat many car trips, even for people that do have cars. Especially if parking spaces stop being subsidized.

I worked for a few years in London, in a place with no tube (Chelsea). Busses were a pain. Slower than walking in most cases.

Massive difference between “I visited and took a couple” and “I rely on them to get to work every day”. I can spin up a few VM’s on any cloud provider in the world and they’ll all work fine. That doesn’t mean every provider will be equally suitable for my production load.

I lived in SF for a period. I was lucky enough to be able to walk to work, but my roommate had to take two busses both ways. At least once/week one of those would go offline and they’d have to walk.

The problem with buses isn't that you can't use the ones that exist, it's that the ones you need don't always exist. The bus goes where it goes but not everywhere.

Then you have to go places the bus doesn't go often enough that you need a car, and once you've already paid for a car and taxes and insurance, it costs more to take the bus than the incremental cost of driving everywhere.

"once you've already paid for a car and taxes and insurance, it costs more to take the bus than the incremental cost of driving everywhere"

I agree and mentioned this in my other comment, but I'd also add that in my experience there are arbitrary* gaps in what routes are available.

Nearly all the buses in the city where I live go west to east, from a certain midpoint to down town. But I live about 10 minutes south of a point on the western side and there are really no routes that go north-south here. And I work for a major employer.

*Seemingly arbitrary to me, but of course they might rationally determine there's not enough riders.

I used to live 5 miles from work with a straight shot down a bus corridor. It was cheaper to drive it. Occasionally I'd set out for work by bike at the same time the bus arrived at my stop and still beat them there.

A typical city bus weighs 8-10x more than of a car and causes a multiple of that in pavement wear. There are many cases where a car would be the more efficient option. The only question is what the car is doing when you aren’t in it.

> when I visited I took several and they were consistently good, and were never empty, just the frequency could be a lot better. They were a lot noisier than the buses in Europe, though.

If "noisy and infrequent" doesn't mean "not good" to you, what does?

Taking a bus would add anywhere from 15-45 minutes to my ~5mi commute, with the uncertainty mainly due to the infrequency. That's not good.

> If "noisy and infrequent" doesn't mean "not good" to you, what does?

It's inconvenient / not ideal. I took a loud, full, public transport bus to school in my home town every day. It was not always on time. It got me to the destination every day though.

It was not great, but it was good enough. Sure, a car would be better, but if everyone did that everyone would be even later and busses would be even more infrequent.

Buses are a great place to catch airborne disease. As a whole, the deal with public transportation is a lot less rosy than some of the proponents would like one to believe.

Buses is unreliable even in Japan because it uses jammed public road, so we avoid using buses.

My main issue with bay area public transportation is the public. I've had a person threaten to stab me because I was white. He got away. I've had someone smoke meth and then scream in my face, wanting to fight me because I was being racist. He got away. I've had someone pull out a taser and start playing with it. He was arrested, fortunately. One of my acquaintances has been robbed at gunpoint. Not sure if they ever caught the guy.

In addition to the risk of violence, there are also the annoyances. Smelly homeless people. People smoking. People blasting music. (The buskers are the worst.) In a car I have none of those issues. I don't even have to listen to someone loudly talking on their phone. It's quiet. It's clean. It leaves when I'm ready. It's far less stressful.

Nowadays there are only a couple of commuter bus lines that I'll take. Though with the lockdowns and the curfews, it's been months since I've used them.

More specifically, the problem is that public transportation is public and therefore can't or won't enforce the standards for ridership that any sane private company would--the violent, the antisocial, the mentally ill, the people who believe personal hygiene is optional. When my girlfriend of a few years ago was forced to take the bus to work regularly, she was harassed on a daily basis. Public availability doesn't have to mean no accountability, but somehow it devolves to that. And as long as that holds, folks who have the means will pay to avoid it.

There's something wrong about this. I grew up in Scotland where we had plenty of smelly, drunk, drugged and violent people, but they'd be dragged off the train by the police.

Police have been ordered by the city government to avoid enforcing certain laws. It basically means that homeless people can get away with theft and any nuisance. Also criminals are quickly released re-offend.

Here are some examples.

This guy has been arrested, charged, and released on his own recognizance five separate times. None of the cases have been closed yet: https://twitter.com/SFPDTenderloin/status/123955010416546611...

This guy was arrested, charged, and released without bail several times before he hacked someone up with a machete: https://twitter.com/SFPDTenderloin/status/122409012050613452...

This guy has been breaking into cars for over a year. He was caught multiple times and released on his own recognizance: https://twitter.com/SFPDTenderloin/status/121391498699874713...

Most of the bay area is like this and criminals know it. It's hard to describe just how bad property crime is here. Everyone I know has given up reporting when their bike is stolen or when their car has its windows smashed. They know the police won't do anything.

Or this guy, in Seattle, charged three times for violent assaults and released because he was deemed not competent to stand trial before attempting to throw a woman off a 40-foot overpass into rush hour traffic: https://www.seattlepi.com/local/crime/article/Charge-Man-att...

> the problem is that public transportation is public and therefore can't or won't enforce the standards for ridership that any sane private company would

It is weird that most of the world seems to manage that just fine. At least in Germany they also have no issue with calling the police on misbehaving passengers.

And if you are female, you also get to deal with perverts trying to grope you [0].

[0]: https://old.reddit.com/r/TwoXChromosomes/comments/f92kp7/pub...

So this is a great point - but - we're not bound by AI ... can we not start to narrow the issues with regular tech? For example, in this situation, something that more predictably tells the driver that there is 'literally something in the middle of the road', or at least possibly trigger an alarm?

I wonder if someone can comment on the relevant advances on giving AI the proper human-designed boundaries that may help?

> I wonder if, in fact, we cannot construct a NN controller with the level of unusual object detection and edge-case scenario recognition required to match an average human drivers capability to recognize and react appropriately to unusual situations

Well, I mean, according to some philosophers, our existence is proof that it can at least be done in the physical world...

> to make it one-log better you need 100x the data and 100x the NN dimensionality.

I'm not sure why you think so. At some point the dimensions will get diminishing returns. Maybe you need only 100x the data at this point (or maybe not). But assuming it's both without knowing the current configuration / limits is just a random guess.

Isnt the point that you have to get deep into the diminishing returns to root out extremely rate errors?

Not always. Sometimes the fix may be just to add some teaching samples you haven't seen before and the NN is already capable of handling.

We need remote work, more home delivery options (e-truck/vehicles), greater incentive for electric bikes, and better/more public transportation.

100% agreed. Unfortunately improving transportation and mobility is pretty much orthogonal to profiting off the ai/ml hype cycle.

I don't think its the wrong problem. If we had perfect self-driving today it would still make sense to have all the things you mention, that is more remote work, etc, but you would also have safe long distance cargo hauling, safer road trips, cheaper home delivery, etc.

The technology is called “autopilot”. Blame the expectations of consumers for safe use on Tesla marketing.

All previous mentions of autopilot were for the exact same technology that Tesla uses: assist the pilot but require him/her to keep monitoring its action and intervene when necessary. https://en.wikipedia.org/wiki/Autopilot

Ahh yes, people are definitely more familiar with aircraft functionality than they are with hollywood buttons that do magic, tesla definitely did not imply that the car could drive itself.

I think it was trained (or learned) to avoid the rear of vehicles, and slow down when it sees it. The rooftop doesn't look anything like the rear.

Surely a Tesla has a front radar to measure distance though, no? If it's rapidly approaching to an object that's not moving, who cares what it looks like?

Can it actually distinguish reliably between an overnight truck and a manhole cover, an overpass, and an overhead sign? These are far more common occurrences.

Reportedly it can't.

https://www.wired.com/story/tesla-autopilot-why-crash-radar/

No, but I mean all cars for some time now have to include anti-collision radar as standard, it doesn't use cameras to recognize anything, it just goes "you're approaching a stationary object without slowing down - SLAM BRAKES". I'm guessing what's happening here is that Tesla is aware that it's approaching a stationary object, but the camera system overrides it because it doesn't recognize it as a danger. So in a way, a much simpler system would have prevented this accident,.

You can do that, but what level of false positives are you willing to accept?

How often are willing to have your car perform an emergency stop for an overhead sign? How safe is it to have your car brake aggressively in a middle of the highway for nothing?

I don't know why you keep mentioning the overhead signs? The anti-collision radars that are mandated in every new car are mounted on the bonnet level, they won't react to an overhead sign. And yeah, emergency braking in the middle of the highway is not safe at all, but seeing as literally all new cars have to have that system and we're not seeing people randomly braking in the middle of the motorway for no reason I can only guess it's fine. Besides, the system doesn't start braking if you're actively applying throttle, it will start beeping at you but it doesn't start braking until you lift your foot off the gas.

Because overhead signs were mentioned as one of common sources of radar false positives during the reporting about the 2016 Tesla crash.

> but seeing as literally all new cars have to have that system and we're not seeing people randomly braking in the middle of the motorway for no reason I can only guess it's fine.

But we definitely do.

https://www.cbsnews.com/news/automatic-emergency-braking-in-...

The price we pay for having relatively few false positives is a high false negative rate. For example, a Tesla won't stop for an overturned truck or the pedestrian standing next to it.

>No, but I mean all cars for some time now have to include anti-collision radar as standard,

The way these work is by tracking other moving objects around you rather than stationary objects you are approaching.

It'll track another car to see if it emergency brakes or changes lane too close to you, but it won't track a parked car on the road that you side swipe or head into etc.

The Subaru’s camera system definitely tracks stationary objects, and it’s obnoxiously sensitive. The vehicle throws a beeping tantrum when approaching steep hills or roundabouts at any speed above a crawl. It will auto-brake for a birthday balloon in the road. Pretty sure this overturned truck would set it off.

If you watch the video, it looks like it did slam on the brakes right before collision. Maybe that was the human driver intervening, or maybe it was last-minute emergency system kicking in.

Any indication yet that auto-pilot was active?

While it is true about NN, we should not forget the same is true for human drivers.

the human is the existence proof of a NN that can match a human.

Reply on news.ycombinator.com