Hacker News Reader

Comment by sudosysgen | Hacker News Reader

A big part of ML in robotics is actually making nearly 1:1 simulations of the world and the actuators.

It's fairly sucessful. We can simulate for example driving a car really well.

Simulating human behaviour is harder, but simulating brick laying is not that hard, we have the technology to do so already.

View on HN

I suppose it all depends on the abstractions you make and how well those abstractions hold in the real world. Humans of course make abstractions but many of these are done subconsciously.

Simulating brick laying might be able to be done in a controlled environment, is it possible to make it low cost enough and accurate enough for all general purpose brick laying situations? Probably, given enough investment we could get closer. Is ironing out all the nuances cost effective? I don't know.

We can definitely simulate a driving environment but I given the recent struggles of self driving cars I don't think I would say that we are at the point where we've solved the problem of actually driving them in everyday situations.

The biggest issue with self driving car isn't actually getting the car where we want it to go, it's predicting human behaviour and dealing with unseen conditions. But the actual physics of driving cars, yeah we've gotten them down beyond everyday situations.

>We can simulate for example driving a car really well.

Can you qualify this more specifically? In many domains (particularly safety critical ones) “reasonably well” may not be sufficient

Self driving hours having a lower incident rate than human drivers. That’s the minimum. Having a 99.9x% success rate (put as many 9s as you desire) is your qualifier and that’s a standard measure in operational uptime.

I've heard the opposite of this in that self driving cars underperform compared to human drivers across the board. The sample size of self driving cars doing everything a driver does is also miniscule. Do you have a source?

So far Google's self-driving cars have vastly fewer accidents than humans have per mile. Or what kind of performance are you interested in? I am sure, they could also be made to drive faster than humans and still be safer on average thanks to superior reflexes and foresight.

But what do you mean by 'across the board'?

As far sources, a web search gives many articles about safety of self driving cars. See eg https://www.wsj.com/articles/self-driving-cars-could-save-ma...

You are probably comparing accidents of self-driving cars in optimal conditions (since they cannot even drive in any other conditions lol) to all accidents in all conditions in human drivers.

Humans get into plenty of accidents under clear skies, too.

See https://www.forbes.com/sites/bradtempleton/2020/10/30/waymo-... which tries somewhat to correct for conditions.

I think there are some potential flaws with the “per mile” or “disengagement” metrics.[1] It feels very much like using LOC as a measure of software quality. Sure, it’s a metric but probably not a very good proxy for what we’re after.

[1] https://www.automotivetestingtechnologyinternational.com/ind...

Sorry for the delay. I don't get notifications. Not sure how people are so active on here with communication. Maybe you or someone could recommend a way?

Anyways:

> But what do you mean by 'across the board'?

By that I mean across all of the people driving in the US and their rate of incidence. For example, average miles driven, the amount of drivers and the rate of accidents. I think the most intriguing detail could be drawn from the rate of fatal accidents, since that's the most concerning, ignoring accidents that cause a casualty as I don't know the method for gathering that data off hand. One could glean a lot of info from that. Here's some rough numbers I gathered, and please forgive the naive approach to my data gathering to express a point:

Average miles driven/person[0]: 13,000 Average fatalities/year[1]: 37,000 Approximate number of licensed drivers[2]: 231,652,000

I don't have numbers for self-driving cars and the number of accidents, but regardless, would it perform the same with the same number of miles driven per car. Keep in mind that self-driving cars currently aren't navigating in all circumstances and will beep to make the human take control again. At least with Tesla.

[0] In 2019, there were almost 229 million licensed drivers in the United States. (Source: https://www.asirt.org/safe-travel/road-safety-facts/) [1] Over 37,000 Americans die in automobile crashes per year. More than 90 people die in accidents every day. (Source: https://www.thewanderingrv.com/car-accident-statistics/) [2] https://hedgescompany.com/blog/2018/10/number-of-licensed-dr...

Just to be clear: when I say self-driving, I mean whatever Waymo is doing. Tesla has assisted driving at best at the moment. Waymo is aiming for true self-driving.

Looking at fatalities would make the analysis easier, but when I last checked, Waymo hadn't driven enough miles to make a good comparison possible on that metric.

(They haven't killed anyone yet, but neither would the average human driver have done so, yet.)

So we would need to look at less dramatic accidents.

Yes I think what you're saying is accurate. My apprehension is the lack of concrete data we have for comparison at the time to whole heartedly put my life in the hands of engineers, for this particular thing. I do, however, look forward to a well vetted, tested and regulated, automated driving future.

Car simulators are accurate enough that F1 drivers drive more in simulators than in practice laps. They are very accurate. More than well enough to train a model to drive, for example, reasonably fast. Of course the real world is always different even if simply because the conditions are different, so you keep some headroom.

I think this has more to do with cost and convenience more than being a better representation of what it's actually like driving on the extremely varied state of a track. You can see this with the practice laps being extremely important because of the way the tire compositions, weather and car set ups change the dynamic of the cars drastically. The simulators can't replicate this effectively.

Reply on news.ycombinator.com