After 48 hours we haven't seen any sign people are using Waymo's service
arstechnica.com
arstechnica.com
Even if you had a full-autonomous only Interstate where each car could be fully loaded and ride bumper-bumper at over 100kph, it wouldn't even begin reach the carrying capacity of currently existing fully autonomous subways systems like those in Singapore.
https://penguindreams.org/blog/self-driving-cars-will-not-so...
These driverless cars would be great in Europe where they can solve the last leg problems in countries that already have strong public transport infrastructure. In places like America, the existing infrastructure has a long way to go before autonomous cars won't lead to the same gridlock as normal cars .. and an even longer way to go before they're ready for mass consumption.
Citation needed.
> Even if you had a full-autonomous only Interstate where each car could be fully loaded and ride bumper-bumper at over 100kph, it wouldn't even begin reach the carrying capacity of currently existing fully autonomous subways systems like those in Singapore.
Citation needed.
So do roads, slow downs happen when cars get on or off of highways.
Going from a road with a high speed to one with a lower speed, if those roads are at capacity, will always result in a backup. Off ramps serve this purpose of course, they basically convert cars going at 60mph into traffic going 30mph through stop lights or some other sort of flow control, but if the roadway is at capacity (as many major cities are), that off ramp as a flow control mechanism falls apart and cars pile up onto the freeway.
The math is unfortunately independent of self-driving cars. If every car is self driving capacity does increase, cars can self coordinate and some of those stop lights become unnecessary, but if 100 cars want to get onto a road that has room for 20, someone is going to have to wait!
Subways naturally don't have this problem!
Roads also allow for multiple lanes. This allows for somewhat degraded service when something goes wrong; a train stopped on the tracks blocks everything. With multiple lanes, roads can be maintained while still in service. Every subway struggles with maintenance windows.
I'm not saying one is always superior to the other - trains have their niche, but it's a niche that will never be as broad that of autonomous cars.
What I don't understand is why do autonomous self driving cars and subways have to be either or. Why can't we do both? After all, we can't use the subway for ambulances and fire trucks so we still need to build some kind of roads even if we had full confidence in the subways...
Just thinking out loud... Unless we had helicopters for emergencies?
Can we have a town with no paved road? Only sidewalks? That's still paved, right? Can trash collection also take the subway?
Not necessarily.
Rail is cheaper to maintain, and there is a huge opportunity cost of paving over a cities most valuable real estate to make roads.
The math on how large the streets would need to be in downtown Seattle to alleviate traffic end up with a giant % of downtown Seattle being torn down and paved over.
The Urbanist has a good article (https://www.theurbanist.org/2016/10/12/brt-is-not-cheaper-th...) on why light rail is cheaper than a huge freeway.
Rail can also have multiple tracks, it costs more, but amortizes really well.
Not having to intermix pedestrian, bicycle, and cars is another huge win for, well constructed, rail projects. Autonomous cars still have to stop at crosswalks! In urban cores, stop lights seriously cut into average MPH for a trip, and start shifting things into favor of rail.
Moving 100kph continuously bumper to bumper, not having to stop to let passengers on or off, my bet is that the interstate has more throughput than a subway train.
> Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.
Handling 30k passengers per hour is manageable for most rail lines around the world.
Torontos Bloor line does 500,000 daily.
Similarly highways are constrained to their own maximums per lane, and counterintuitively raising the speed limit doesn't help. 1500-1700 vehicles per hour is a fair estimate.
http://buckholztraffic.com/UNF/Data%20Analysis/2%20Lane%20Hi...
Asking for citations might be a shallow dismissal, but stating hypotheses without facts to substantiate them is also shallow.
There are other ways to ask for more information, that open discussion up rather than stiffening into confrontation. Please post in that spirit instead.
So a highway filled with self driving cars zipping along bumper to bumper may not technically be able to carry as much as a fully loaded subway train, but it's so much more useful. Take away the casualties and pollution, and it's as close to perfect as we are likely to get.
Because, to me, it sounds like you just don't want to be uncomfortable, and that's fine, nobody does, but it doesn't make something more useful.
And yes, before you say it, that isn't an everyday occurrence. But jackasses refusing to take their bag off the seat so you can sit down is. As are people who can't be bothered to take a shower, or think that cologne will make up for it. The list goes on. Many people are just unpleasant to be around, and I don't see any problem with that observation.
For me, public transit comfort comes down to a couple things. First, the hard plastic seats make my butt hurt, and standing makes my left foot ache. Sorry, I'm getting old :). Second, the 2x time it takes to get anywhere means I'm on that terrible seat for even longer. And third, because yes, it's absolutely true, I find that perhaps one in ten people is unpleasant enough that I want to be out of visual and/or aromatic range.
Yeah, same for me. And I've heard that the ~80s crop of NYC subway cars were made in Japan. Except there, they have upholstered seats. But that wouldn't work out well in NYC, so they got hard plastic. But on the other hand, the oldest LIRR cars (refurbished for the 1966 World's Fair) had upholstered seats, and they were ~OK.
Similarly, never have I been given religious literature or panhandled. This happens about 90% of the time.
Also people seem to focus exclusively on the negative interactions which is a very human thing to do, but this discounts the positive interactions.
It can be really nice seeing people offer their seats to the elderly, or kids meeting and playing, or the occasional furry companion in a cute bag.
But I enjoy people watching in general. It can be neat to try to imagine a persons world and how it looks.
Except in lucky places with great underground railways you basically have awkward last-mile journeys where you have to take a taxi or hit a car or take a bus and that adds a lot of time to the journey.
It would be great if there were trains everywhere but frankly that's even harder than making a reliable driverless car.
If I have a car, I load everyone and everything up, and we go. We get to our destination and unload, ta-da! Is this not an order of magnitude more useful than taking a train?
Sometimes I think HN is filled with young people with uncomplicated lives :).
No, if these are big issues for you, you likely live in an American low density area where public transportation is scarce. In large metropolitan areas (which is after all, where most of the world already lives, and will live), getting on and off public transport is usually not a problem.
I have relocated to Tokyo for example about a year ago, and for me, a trip to the grocery store is usually done by foot (same is true for other appliances), and multiple forms of public transport are in walking distance.
With all due respect, the number of people who live in cities as dense as Tokyo are the minority, most of us don't.
However, when I think about loading up my young children with all of their stuff, loading me up with everything I need, etc, and then dragging that down the street a quarter mile, then waiting for the bus, boarding it, riding to the transit center, transferring, riding another bus to the light rail station, riding that downtown, and then doing it all again the other direction... well, I'm pretty confident in my original point that the car is a hell of a lot more useful. Sure, you can exist with only the public transport option, but I was not asserting otherwise. But a lot of people on HN seem to think that not only is it possible, but that it is the superior choice.
So, no offense taken, in fact I appreciate you affirming my original comment about so many folks on HN commenting from the viewpoint of someone with a young, uncomplicated life :-).
As it is those with cars push off these negative externalities onto those who choose not to have them, or can't afford them. So your comfort does have a cost, which you don't pay directly.
Also, tour last paragraph just makes you sound pompous and liek you have no intention of having a discussion but instead only wanting to put others with a different point of view down.
I exclusively use public transport, and regularly do both of these without any problems.
I imagine (edit: many) people who use public transportation would agree with you. You now, since they're usually using it out of necessity and not enjoyment.
Right. We get that. It's very obvious that much of the hype around self-driving cars is premised around people who simply reject mass transit outright and therefore believe that somehow, somewhere, they'll get to ride around in a spacious, individual vehicle in the middle of a packed city and so will everyone else and somehow it will all work out because "technology".
Personally I expect self-driving cars to drive a great deal more conservatively than humans as a result, and I think self-driving cars are going to end up demanding a lot more space and buffer on the freeways. And if they don't at first, one day you'll way up and they suddenly all will, because somebody won some lawsuit and a software update was pushed overnight to double the buffers.
I think other drivers will adapt to this and start taking advantage. Think of what happens when you leave a legal following distance in front of you now, in traffic. Now imagine an autonomous car in the same situation. It would probably end up stopped.
And we haven't even considered networking. When the cars are all talking to each and routed together then it's fundamentally a single machine.
The laws and systems will evolve with capabilities rather than getting stuck on supporting the earliest generation of self driving vehicles and norms.
There's still a chance an automated car encounters something that couldn't be spotted / reacted to. For example a deer/kangaroo/whatever jumping right in front of it. If the cars keep their distance, only the first one crashes. If they form a minimum-spacing train, the only message they can send to the following ones is "you're screwed now".
I don't have much experience with deer behaviour, but kangaroos will happily jump right into anything moving. They will even hit the side of your car sometimes (which they should've obviously seen coming)
Subways / trains in Tokyo are faster than driving, same in London.
No stop lights, no intersections, no delays on the road.
Buses aren't faster, but once you go underground, the speed improves dramatically. Even if there is a 2 minute stop once every mile, that is still less time waiting than comparable stop lights in a car.
I think what it comes down to is that if you have a subway system, that will probably beat cars, in the limited area it serves. Once you have to do transfers to buses, it gets much slower. In the places I've visited with subway systems (SF, DC primarily) this is definitely the case.
That's probably US bias, because as far as I can tell by reporting US public transport is generally very bad and far worse in big cities compared to much smaller cities elsewhere.
I look out the window of the train, though, because otherwise I get motion sick. It's lost time to me.
Private point-to-point loses out mostly for long-distance already, short-distance currently either allows last-mile, station overhead, and related concerns to big down shared transportation or suffers from sharing infrastructure optimized for private point-to-point.
Ubiquitous automated (on-demand, not personal cars actually make the station problem easier, since you need less parking at stations, making station footprints smaller, and reducing delay from arrival at station to boarding.
They could also merge with buses: instead of fixed routes and schedules, vehicle sizes and routing are demand driven for efficiency (perhaps allowing you to pay a premium to guarantee a dedicated vehicle for your party.)
Aside from that, I agree with what appears to be your premise -- that we could invent public transport devices that mitigate and perhaps mostly erase the advantage of the car.
But then I'd ask ... why? Why are we supposed to hate cars? It's okay to like scooters and bikes, but nothing larger until we get to a bus? I imagine a future with all sorts of automated cars that never get into an accident and roll silently along emitting not a whiff of noxious fumes, while being big enough to carry just the right amount. If nobody needs to own their own car, then they can call a car that is sized for their exact needs. One passenger or ten. They efficiently get to exactly where they are going in the minimum time possible with minimum effort. What a dream!
That's not quite my premise.
Rather, it is that shared self-driving automobiles (cars through vans through buses) with centralized automated dispatch are the missing element of the mix of public transport devices that, alongside existing options, does that.
> Why are we supposed to hate cars?
Recognizing that public transport has social benefits does not require hating privately owned cars, much less cars more generally.
> It's okay to like scooters and bikes, but nothing larger until we get to a bus?
Size has nothing to do with anything (well, it has to do.with efficiency for long-distance transport.)
The systems could be ride share type shuttles that can pick you up at your location and drop you off at your destination.
You make the request in the app and the shuttle picks people up in a location.
They can even drop you off for a transfer with other modes, other shuttles etc. The shuttle that picks you up doesnt have to be the shuttle that drops you off.
Shuttles like these will greatly reduce traffic and can also handle spikes in demand unlike current mass transit.
Every Australian capitol except Darwin/Hobart and ACT have commuter train systems, and they have the same land mass as the US with 1/6 of the people.
I hate hearing people use the argument you used, because it's just straight up wrong. If American built rail, you'd be surprised to see how quickly development would flock to build around it. American spread out like it did because the nation moved to Interstates.
How do you explain car craze in china. Its the biggest car market in the world and one thats growing rapidly. So exactly the opposite of what you are suggesting would happen is happening in china.
China comparisons makes no sense. Car ownership is not accessible to most chinese they have no option but to build a rail system. It is not because ppl prefer travelling in trains(vs cars) its because they have no other option. You have cause and effect reversed. If everyone in china can afford a car like USA they would prefer American interstate system. Who would possibly want to live in cramped quarters built around rail system when you can live in suburban mansion with basement home theatre system.
Sorry if I sound selfish but it is really scary to think of 1.5 Billion people doing what the 330 million US Americans do.
At some point, I'd argue there is no such thing as sovereignty. I, a neighbor, have a right to not suffocate to death more than you have a right to burn your house to the ground (or I'll take it by force even if I don't have that right).
https://en.m.wikipedia.org/wiki/List_of_countries_and_depend...
Edit: Wikipedia’s article on China says, “Covering approximately 9,600,000 square kilometers (3,700,000 sq mi), it is the third- or fourth-largest country by total area, depending on the source consulted.“ Now I’m just confused. Area of a country seems like something there would be broad agreement on.
Nitpick, but in Singapore only three subway lines are fully autonomous (CC, DT, NE). The remaining two are not (EW, NS) because they were built with older technologies.
There are so many people who cannot afford housing anywhere near Seattle and they are the runs who run your Wal-greens and Starbucks. They have to sit in traffic on I-5 or wait for express buses. This will be a big step up and it's only a start. Seattle needed ST3 a decade ago, and it's going to have a way bigger impact that you think.
This is a poor comparison because cars on highways don't serve at all the same needs as subways in one of the densely populated areas in the world. I'm sure I could construct comparisons where airplanes fail, but a fleet of bicycles succeed(or vice versa), but that doesn't mean that humanity should pick one or the other as the optimal solution.
A fully loaded Toyota Corolla carries five people and is 4.65 meters long, so at 100 km/h each lane of the highway would carry 30 passengers/second. So the highway would carry several times more passengers than the subway.
Even though each train can pack more passengers per square meter than a car, it's not possible to run the trains back-to-back at 100 km/h, because they need to stop to let passengers off and on. At rush hour, the rate of the subway system is basically limited by how quickly people can flow from the platform into the train.
Though the problem isn't just throughput of the freeway, but also a question of where all of those cars are going to go when they get to the destination -- generally the commute traffic is going the same place at the same time. Self driving cars may not need a parking space, but if they are going to drop off passengers and then drive away to park somewhere outside of the city center, that causes even more city traffic.
because they need to stop to let passengers off and on. At rush hour, the rate of the subway system is basically limited by how quickly people can flow from the platform into the train.
That's why large train stations have multiple tracks.
> We now know that Waymo One won't be fully driverless; there will be a driver in the driver's seat.
This reminds me of this famous Radio Yerevan joke:
> Q: Is it true that Ivan Ivanovich Ivanov from Moscow won a car in a lottery?
> A: In principle yes, but:
> it wasn't Ivan Ivanovich Ivanov but Aleksander Aleksandrovich Aleksandrov;
> he is not from Moscow but from Odessa;
> it was not a car but a bicycle;
> he didn't win it, but it was stolen from him.
Because smart people have invested billions of dollars into this technology and because even smarter people work on this technology the general public must pretend that it's all going according to plan, that everything is damn serious and that the emperor still has his clothes on.
"The man who I am about to introduce," he said, "is someone I know you'll enjoy listening to. He is the most gifted businessman in the country. He has made one hundred million dollars in California oil."
The speaker, embarrassed, came to the podium. "Thank you for your kind introduction, Mr. Chairman," he said.
"However, the facts need some clarification. It wasn't oil; it was coal. It wasn't California; it was Pennsylvania. It wasn't one hundred million; it was one hundred thousand. It wasn't me; it was my brother. And he didn't make it; he lost it."
My game leg has been acting up — we’re in for a bitter winter..
Q: Is it true that the great poet Vladimir Mayakovsky committed suicide?
A: Correct, and his last words were “comrades, don’t shoot!”
This makes sense, because a lot of times merging into a lane is a game of chicken. You've got to make the drivers in that lane believe that you're going to merge anyway and that they'll need to slow down to avoid hitting you.
A lot of city driving, especially rush hour city driving, is about creating an opening. Not about waiting for one.
But yeah if your turn signal is on and you're not moving into my lane within 1 second, I'm assuming you forgot to turn it off.
Exactly, which is why non-city drivers struggle to drive in big cities. They're used to just waiting for an opening.
I had an accelerated course in aggressive driving over a couple of hours when going through it. In comparison driving in SF or NY is like visiting a spa.
Now, driving in southeast Asia or southern Italy was very, very different from any US city.
I really like this video for describing some of the advantages of "zen driving":
you better slow down and give them an opening or you'll hit them. which you will definitely feel like doing sometimes.
that's one reason i really look forward to the day when cautious, defensive, courteous self-driven cars replace all human drivers here.
Me: it's really sunny today
AI: I love sunny days
Me: me too, maybe that's why I'm so happy today
AI: why do you think you're so happy today?
But in all seriousness, IMO the biggest challenge to AI driving is not object recognition or modeling instantaneous state, but the full hierarchy of “statefullness” which humans keep effortlessly, but which seems to be almost totally missing from the driving AI.
Understanding the current state, the upcoming state, and the expected transitions in between... are crucial to how I am able to turn left with oncoming traffic, smoothly cruise through a 4-way intersection which happens to be on a curve, zipper merge with entering cars coming off an on ramp into my highway lane during rush hour, change lanes to avoid a bad driver, and not slam on the brakes when a stopped car is in the process of turning left up ahead.
When driving on a two lane undivided road, and a left turn lane appears ahead, suddenly an AI finds itself driving in between two lanes and makes a random choice to veer left or right to try to stay in lane. Without the stateful understanding of how and why this new lane came into existence the AI is clueless. The AI can’t solve the problem just by identifying lanes at t=0, or trying to read signs or markings on the road. Us humans know intuitively (or from memory) which lane is for onward travel and which lane is for turning. Any AI driving system built in the next 5-10 years will need an annotated map which tells it which lane to choose.
This all adds up, in my estimation, to a metric fuck-ton if metadata required per mile in order for an AI to operate. It cannot be done “statelessly” (like a human can do it) without an AI at least 5 orders of magnitude “smarter” than what we have today.
Asking out of real curiosity, not trying to debate.
What do disabled folks do now in places that don't have such a service?
The only advantage of a self-driving cars is that it might be somewhat cheaper.
Video: https://youtu.be/ArYTxDZzQOM
It sounds like they're taking this seriously.
[1] https://www.youtube.com/watch?v=cdgQpa1pUUE
[2] http://www.visionbeyondsight.org/About-MissionVision.htm
Disclosure, I work for Google, but have no connection to Waymo.
I hope the team can keep pushing on; Waymo is one of the most important "moonshots" to the country. It really could save lives. But, one mistake and people won't trust it even though hundreds of automobile accidents happen every day.
The only thing we might have are dedicated lanes on interstates for driverless cars, like railroad tracks.
In city driving will never happen without AI.
This is a strong claim, but I'm sticking to it.
Able to teach itself anything from unstructured (unprepared) data.
To me this all sounds like some type of weird meatbag chauvinism, where it's absolutely impossible to perform any complicated task that a computer hasn't managed to do a good enough job of yet without resorting to building some sort of Commander Data in our image. And, while that may be true for such human abilities as "being able to do a wide variety of genuinely hard things if they are taught, especially at a really young age" or "arguing about abstract concepts we don't fully understand ourselves", driving a car is...well, it's not that hard. Human beings in car cultures learn to do it, but if those same human beings grew up in horse nomad cultures, they would learn to ride horses and shoot arrows from horseback really well, and if they grew up in a different culture they'd get a different skill. Being versatile enough to develop any of those skills given the right childhood experiences seems much harder than any one of those specific skills.
Of course safety drivers are going to be there for a while. Was the perception really that they would test a bunch and then just pull all of the drivers immediately after? Why would they take such a risk when they can afford to have the drivers monitoring for a while?
It's almost like the tech press wants Waymo to aggressively pursue a very public failure.
Even one disengagement every 5000 miles without human backup will be perceived as apocalyptic when your fleet drives thousands of trips a day.
Where are those social media posts?
Waymo's PR is getting more and more divorced from reality with every announcement.
My take on that is that it's okay. They're being cautious.
If it's after an extended test period, it's not "immediately".
But yes, the perception was that they would test a bunch and eventually pull the drivers out of the typical car. And that they would need to do this before it could be considered a proper launch.
So the car actually behaved correctly then? It's called a zipper merge.
Most of the "mistakes" in the video looked to me more like a hyper-cautious grandmother than an A.I. that fundamentally doesn't know how to drive.
That's my theory as well. Remember, Uber killed a pedestrian earlier this year. Fortunately that didn't result in self-driving cars being banned or regulated out of existence, but they may not a get a third chance, so it makes sense to be even more cautious after that.
I'd say we will find the machines to refuse to drive like humans. And we'll have to adapt.
Of course, there are listed companies with significant AI investments. But I'd argue their valuations are driven by their existing/conventional/ad-driven scale, not their AI arms.
Whether or not they're overvalued remains to be seen. The era of cheap money seems (?) to be coming to an end, so it will be an interesting few years.
It was the IPOs that drove the boom (and subsequent bust). Public investors falling over themselves wanting to get in on the "next big thing". Older generation companies were never going to deliver the kind of quick return multiples that characterized the boom (though as we saw, the enthusiasm certainly jacked up their share prices too, but by and large they managed to ride out the storm).
1. The cost of screwing up is very high.
2. There are important events in the real world environment that are not very recognizable or predictable.
3. Humans accept a level of danger from other human drivers that they absolutely will not accept from robocars.
The challenging cases are probably things like children playing on the side of the road, pedestrians, bicyclists, construction workers, cops standing in the street, people working in manholes, and so on. Relatively small, easily obscured, unpredictable, hard for computers to figure out how they are moving, and very high cost to hit.
Also I think the actual hardware is not good enough yet. LIDAR is great but humans can see really far in really good detail and can work out what is happening even in really bad conditions (rain, looking into the sun, etc).
Non-motorway driving is just too varied. I still think the first real application of autonomous vehicles will be driverless lorries that drive between depots that are connected by motorways. It's way way easier because the environment is so restricted, there's a clear commercial need, and people would be a lot more comfortable about the safety aspect - who hasn't seen a lorry drift into the hard shoulder? Plus lorries are super expensive so it wouldn't increase the cost too much, and they're huge so there is plenty of space for hardware.
If, instead, we have a set of local sensors setup that identified the new "safe" path, an automated car could easily go through this new section. As marked, however, it would be very dangerous to rely on an automated driver.
Speaking as somebody who owns a classic car, drives carefully to baby said classic and often rides shotgun... I gotta say: most drivers are pretty decent. The people I hear complain about how awful everybody else is... they're either old and drive dangerously slow, or they're super entitled and drive aggressively. Since we're on HN, I'm gonna guess you're of the latter camp. Do the world a favor: take Uber and don't write AI.
While it is amusing, doesn't HN encourage people to be civil?
I don't get paid by-the-hour to drive; I get paid to arrive on-time. So commuting in my opinion (like IT in MBAs' opinion) is purely a (time-)cost center. And from that POV the average driver is indeed absolutely terrible. Ever since I started working in the software industry, I don't think I've ever taken the wheel of my car and driven it somewhere, without getting angry at another driver along the way.
Granted, different drivers can be terrible in different ways.
Some of them do stupid mutually-harmful BS like pulling out right front of me with minimal margin and no intent to travel at even half the speed-limit.
Others think that because they have an 8-cylinder engine and a traditional automatic transmission, that they should rule the streets - which they can and do, but that doesn't make me any less angry at them when they e.g. pass me at a 20mph relative-speed on the same side into which I was signalling to change because I was behind a schoolbus.
And no, talking about the avarage human being's ability to operate a specific machine over the course of their life is not painting any individual with a broad brush.
Also, an individual can be a good driver. People in general may not be. That's not a contradiction or an insult. Heck even a "good" driver can drive badly.
Ironically though, Uber is then taking that money and putting it into their own AI so they can get rid of one of their bigger expenses; human drivers.
No, I simply expect others to follow the same laws that I'm bound to and that I follow. The Golden Rule. And a lot of drivers on the road simply do not do that.
> humans are remarkably adaptive and can convey intention through body language -- even through a "puppet body" like a motor vehicle. Humans can, by and large, adapt to "terrible" drivers -- swerving to avoid the aggro asshats
I agree with you here, but this doesn't really help demonstrate that humans are not, on average, poor drivers. That some of us can compensate for the shortcomings of others is great, but the fact that we not only compensate for the shortcomings, but that we're able to suss it out of such fine data — seems only to be evidence that we have a need for it, because collectively our driving is bad. Perhaps you can teach an AI to read the "body language" of a car, but I imagine that such a thing would be remarkably subtle and considerably difficult.
(On the whole, I don't think it really matters what the skill level of humans are. The AI must perform better, or else what's the point? Ideally, we should be able to prove that it's better — so as to sway public opinion, and I suspect that will be the hard part.)
Quite the opposite, on average humans are 99.99999% reliable when driving cars.
Math: https://news.ycombinator.com/item?id=12012166
Computers on the other hand are nowhere near that reliable.
Google AI blog explains why:
Simulating many years of robotic interaction is quite feasible with modern parallel computing, physics simulation, and rendering technology. Moreover, the resulting data comes with automatically-generated annotations, which is particularly important for tasks where success is hard to infer automatically. The challenge with simulated training is that even the best available simulators do not perfectly capture reality. Models trained purely on synthetic data fail to generalize to the real world, as there is a discrepancy between simulated and real environments, in terms of both visual and physical properties. In fact, the more we increase the fidelity of our simulations, the more effort we have to expend in order to build them, both in terms of implementing complex physical phenomena and in terms of creating the content (e.g., objects, backgrounds) to populate these simulations. This difficulty is compounded by the fact that powerful optimization methods based on deep learning are exceptionally proficient at exploiting simulator flaws: the more powerful the machine learning algorithm, the more likely it is to discover how to "cheat" the simulator to succeed in ways that are infeasible in the real world. The question then becomes: how can a robot utilize simulation to enable it to perform useful tasks in the real world?
The difficulty of transferring simulated experience into the real world is often called the "reality gap." The reality gap is a subtle but important discrepancy between reality and simulation that prevents simulated robotic experience from directly enabling effective real-world performance. Visual perception often constitutes the widest part of the reality gap: while simulated images continue to improve in fidelity, the peculiar and pathological regularities of synthetic pictures, and the wide, unpredictable diversity of real-world images, makes bridging the reality gap particularly difficult when the robot must use vision to perceive the world, as is the case for example in many manipulation tasks.
https://ai.googleblog.com/2017/10/closing-simulation-to-real...
https://blogs.nvidia.com/blog/2018/12/03/physx-high-fidelity...