Hacker News Reader

> When they want data about a school intersection in SF at a certain time of day, they just... synthetically generate it and simulate

I think it's more about detecting changes to the world. You need boots on the ground, so to speak, to see that new speed limit sign or the new lane paint. The Waymo vehicle can no doubt react to changes in the world when it encounters them, relaying them back to the mothership, but it's better to know about them in advance.

Most AVs, definitely Waymo vehicles, are self mapping. They can detect environment changes and relay it to the entire fleet. That's because they map using the same vehicles as the fleet.

>You need boots on the ground, so to speak, to see that new speed limit sign or the new lane paint.

It'll shock you to know that you can simply get this from governments, some even provide this in API form

> or the new lane paint.

I'd be surprised if this is a thing outside the biggest US (and European, for that matter) cities, judging from Google StreetView there are lots of streets in US cities/towns with almost no paint lines at all.

Do you mean in the API? I live in an European country and I don't think I ever saw an asphalt road without paint lines. This varies a lot between countries though.

Many American roads don't have lines. Residential roads, parking lots, many business driveways have limited markings.

Then there's roads with just the center line markers with no road should markings.

Then there's a whole class of roads of lines over "demarked" old lines that weren't demarked well, or lines fading that should've been painted a long time ago.

I'm surprised you've never seen a non-perfect road?

Here in Bucharest there are quite a lot of big boulevards that do not have them, either because they haven’t been repainted over in a long time or because they laid new asphalt without bothering to repaint the lines (this happens a lot, unfortunately, and is very frustrating).

Does Italy count? https://upload.wikimedia.org/wikipedia/commons/0/0c/Google_M...

Small country side roads routinely lack a central line in Sweden. Even smaller roads can lack the side lines too. And I'm talking asphalt roads here still. The same happens on many residential streets in towns and cities.

But sure, it would be rare to have a large road or street without markings. But most roads aren't large. Most travelled kilometers happen on large roads, but that is not the same thing as most roads. And many individual journeys would involve at least a little bit of small roads at the beginning, end or both.

And of course, if they are covered with snow and ice during the winter you can't see the markings anyway.

It probably won't shock you to know that those sources of data can be months to even years delayed from what's actually out in the world.

no visual data, you need picture data for that. companies like NC tech do it for like $1m a city. or thereabouts.

That’s dumb then. It shows it’s just brute force rather than AI.

A human doesn’t need to be shown every single road that exists in order to drive.

Just a bunch of sophisticated if statements, I guess.

That's true, but the human can do a much better job planning for the journey if they know what to expect along the way.

One example, from the end of the journey: knowing in advance where the actual entrance to the business is, or the specific curb cut that leads to the residence, makes it easier and far less error prone to decide exactly where the journey should end. Even humans have a hard time figuring out the right access point for a business or residence. This is a job for an offline process, fed by as many data sources as possible.

Yeah I'm not so sure this CTO is on the mark here, but to be fair, I do think some of this IRL long tail/edge case data is important for Waymo. The simulation software is super interesting to me - the real world can be so chaotic, and even if they could generate every possible real life case, there needs to be validation on whether the Waymo driver is responding in the optimal way. They certainly haven't solved this problem, you can see some of their growing pains in all of these articles - floods in Austin, more and more interactions with emergency vehicles that first responders seem to believe are getting worse, etc.

Tesla on the other hand has billions of miles of data, yet because there is a limit to camera-only techniques, that data isn't that useful is it? They have no ground truth data to evaluate their camera system on, which is why sometimes you see those Teslas driving around with lidar rigs mounted on them. Going camera-only is just asking for trouble.

I agree real world data is important for Waymo. I didn't mean to say it wasn't, so I've edited my comment to reflect that. It's just that data is not some magic bullet to achieve self driving like Tesla and others suggest.

Of course, Waymo still has much more room for improvement. But it's much more efficient to supplement less but higher quality IRL data with large amounts of synthetic data, than to run a million data collection vehicles 24x7 because most IRL data is boring and useless.

Waymo said 6 years ago they simulate 20 million miles every single day [1]. Clearly, it's working for them given their scale of deployment right now.

[1] https://waymo.com/blog/2020/04/off-road-but-not-offline--sim...

Although most of the real-world data is probably boring, collecting more of it likely makes discovering rare edge cases more likely. But since they happen rarely, I imagine that after discovering them, they would then need to figure out how to simulate them.

> The best example of why data collection isn’t the bottleneck is Tesla.

Exactly. plus any delivery company/dashcam company can provide a bunch of data where ever there is any sizeable population.

About 8 years ago, that data would have been really valuable, but at best its nice to have.

the only thing that is valuable is the breadth of different cars, but even then its not that much of a differentiator.

The biggest difference, is Uber has vehicles around the world. So there's more data from countries with different rules from the US. Signage is definitely different between the US and Europe.

I.. am amused by the confidence on display, but I can't say that I am not concerned that people are confidently stating that real world data is not useful, because it can be just simulated. One would think that, by now at least, we know that simulation is at best an imperfect copy.

And I don't like the idea of even more data being harvested and used.. I just find the dismissal.. odd.

“Real world data is not the bottleneck” != “Real world data is not useful”

No one is suggesting the latter.

Parent's post noted that it is not a bottleneck, because it can be readily simulated ( and thus not useful ). I am not sure if QED is too much in this case, but I stand by my amusement. Or are you arguing that real world data is somehow less useful than simulated data? It is very confusing. I would accuse of nitpicking, but I just noticed you are the parent:D You can certainly speak for yourself.

> and thus not useful

Again, I’m not suggesting this. Bottleneck has a specific meaning. It means Waymo is limited by not having the ability to collect data. Well, clearly that’s not true because Waymo already has a reasonably scaled deployment across a dozen cities that no one else has and can handle millions of scenarios.

Real world data is absolutely required, but more of it doesn’t give you magical self driving ability as Uber’s CTO suggests. If it were the case, you’d see Tesla achieve fully autonomous driving years ago.

I accept your argument. I may have been a little too nitpicky and you do have several good points.

> The best example of why data collection isn’t the bottleneck is Tesla. They boast about billions of miles of data, yet they’re struggling to put out fully autonomous vehicles.

Well, TBF, the tesla data was complete garbage with earlier vehicles. They had cheap and somewhat bad cameras in the earlier vehicles that was only somewhat recently updated. And even then, I don't think Tesla is at the end of their hardware journey. I think they don't think that either, which is why they've gone to a subscription only model for self driving vehicles.

Waymo, on the other hand, has gathered less data, but more high quality data. They do the expensive mapping of a city which is a big part of why their vehicles have early on been able to do some pretty impressive feats. The drawback is getting that high quality data takes a lot of time and resources.

> And even then, I don't think Tesla is at the end of their hardware journey.

I dunno about that. Tesla seems completely adrift, pretending to pivot with random forays into humanoid robotics or whatever, to the point that I wouldn't be surprised if they exited the consumer vehicle space altogether within the next decade. They have no answer for Chinese competitors.

Well, let me rephrase, the previous stated goals of Tesla around self driving cars isn't complete with the current hardware.

I recently watched some videos related to the production of cybercab, which has now started public testing. They’ve still done some great engineering, to the point that the car is now assembled like a matchbox car. All the drive components are contained in a single package for a FWD configuration that the body just drops down on. The car now has no controls besides the screen and door pulls. The materials are all lower cost and they even found a way to skip painting the cars. All of this should help them cut costs significantly.

As far as the self driving, they may be far off still, it’s hard for me to get a read on that and this vehicle is a bet that they will be able to achieve it - right down to the braille in the cabin, so maybe that’s why they still fail. The thing I will say is that despite the PR disaster that the CEO is, which gives us that feeling that the company has lost its mind, it seems they are still quietly doing some advanced engineering.

Didn't they need the data from the 200 million miles or so from actual driving before they could get to the generative model though? Data isn't everything, as you point out with Telsa (mainly because they decided to forego using lidar it would seem), but it is pretty fundamental.

IIRC, they had clocked 20 million real world miles before starting to scale their deployment. But they were also driving 20 million miles in the simulator every day: https://waymo.com/blog/2020/04/off-road-but-not-offline--sim...

> before they could get to the generative model though?

Is that the right kind of model for this particular application?

Waymo might very well be missing specific kinds of data (e.g more incidents/accidents, near-collisions etc)

Also, Uber’s data might be useful for eval, not training (e.g « here is how Waymo would behave vs human drivers therefore it is safer »)

> Waymo might very well be missing specific kinds of data (e.g more incidents/accidents, near-collisions etc)

Accidents and near-collisions are exactly the kind of scenarios perfect for simulation. You don't test them out in the real world and risk injuries/deaths. You need to have confidence they're handled before you deploy.

Again, how do you know you've handled it correctly without ground truth? Simulation without ground truth is a garbage in garbage out situation.

I find the idea of learning from simulated data so unintuitive. How can you radically improve your model with just your model? I take it people do it, so it must work, but i just don’t understand it at all.

Well there's a world simulation model and then the driving model.

You can imagine improving i.e. a specialized math model (problem in, theorem out) with a normal LLM that knows lots of problems and theorems generally.

I think people are skipping over the fact that Google has had cars driving around taking photos for 20 years. I imagine that was used to build the world model in the first place.

They're two different models - you can use the world model to train (or test like Wayve) a different car-driving model.

The world model is basically intended as a more true-to-life simulator.

"You can’t be the CTO of Uber wanting to do AVs, and get the data collection requirement shockingly wrong."

Problem 1: Cost and privacy constrain limit data collection.

Problem 2: It makes not much sense to collect and store data that you already have. Yet you don't know that when collecting if it is useful or not.

Problem 3: P2P in urban setting fails at edge cases which by definition are rare to collect.

All of these problems limit AV scaling.

Yes, the way to make these things safer is to make up data and simulate on that.

Do you hear yourself?

That’s literally how it works right now, so yeah.

>Mapping out every intersection, sign, and signal Before our Waymo Driver begins operating in a new area, we first map the territory with incredible detail, from lane markers to stop signs to curbs and crosswalks. Then, instead of relying solely on external data such as GPS which can lose signal strength, the Waymo Driver uses these highly detailed custom maps, matched with real-time sensor data and artificial intelligence (AI) to determine its exact road location at all times.

https://waymo.com/waymo-driver/

That AI part is doing a lot of heavy lifting. They're using real data. We already know synthetic data is dangerous. Explains a lot of if you think it's more reliant on that than real data.

Mapping and simulation have very different purposes. Doesn’t look like you’re familiar with the basics of AV technology. Explains a lot why you’re confused about how real world and simulated data is used.

the word synthetic says it doesn't exist in reality.

Do you know anything about engineering?