Hacker News Reader

Comment by jeffbee | Hacker News Reader

The Waymo approach has proven itself. Targeted testing, running the driver in hard mode as much as possible, is better than the Tesla approach. Tesla’s plan was essentially “machine learning underpants gnomes”. Collect ten billion miles of data from guys driving down 280 -> ??? -> profit. Didn’t work and now Tesla is recognized as being dead last in the game, behind dozens of no-name Chinese players.

Now what I wonder is whether Waymo has a durable lead over Cruise, or if Cruise succeeds alongside them.

View on HN

I have always been a huge proponent of Waymo and obviously Tesla doesn't have something comparable to them now.

But if you look at videos of recorded Tesla autopilot drives on Youtube you will see a huge amount of success in terms of getting people to their destination without crashing or killing anyone. And there is constant gradual improvement.

If Tesla can get another 2-3% improvement, they may actually be able to launch a robotaxi service. The trick will probably be that you need to have a driver's license and sit in the front seat at first. But after X months they will probably allow sitting in the back, especially for certain vetted trip regions.

> The trick will probably be that you need to have a driver's license and sit in the front seat at first

Rental cars? Lame.

> If Tesla can get another 2-3% improvement, they may actually be able to launch a robotaxi service.

2-3% improvement in what metric exactly? Disengagement rate? Tesla has a long way to go there in anything but highway driving.[1]

YouTube videos are not proof of the safety of an autonomous vehicle. Tesla autopilot is currently very optimistically at level 2.5. 2-3%, even across all metrics won't get them to level 3, much less level 5.

I don't blame the people working in their self-driving program. The expectations that have been placed upon them are unreasonable given the constraints on equipment cost and aesthetic design.

1. https://twitter.com/TaylorOgan/status/1458169941128097800

2-3% is not nearly enough. They need to reduce fatal/serious accidents by like 99.9% from where they are.

Tesla's advanced-driving systems (AP and FSD) lead the world in terms of deaths. It's not even close: Tesla has more fatalities than every other advanced-driving system in the world, combined.

Even massaging the data in the light most favorable for Tesla, a Tesla is still 3x more likely to be in an accident resulting in injuries than the next worst advanced-driving system, and is more than 50x more likely to be in an accident causing injuries than Toyota's or GM's advanced driving systems.

A 2-3% improvement still means that Tesla is the most dangerous advanced driving system on the roads.

Do you have a source for these claims?

> Toyota's or GM's advanced driving systems

What advanced driving system? Do you mean lane assist with adaptive cruise control? That’s not even remotely comparable.

It doesn't matter what advanced driving systems you compare; Tesla is the world's worst in terms of accidents, injuries, and fatalities. For sources: NYT and WaPo both have a series of articles covering this; the NHTSA has released a report documenting over 100 Tesla advanced driving accidents.

It's not 100. It's 100/X and if you don't know what X is then your number is worse than useless.

It's 19 fatalities in the U.S. and over 273 accidents as of June 2022. The numbers have gone up since then. (https://www.cnbc.com/2022/06/15/data-shows-tesla-accounts-fo...).

For comparison, the entire rest of the world's automotive industry has 1 fatality and fewer than 100 accidents, despite having advanced driving systems on more than 3 million vehicles.

But nice try, I'm sure Elon would love to hire you hire Tesla's damage control department.

> It doesn't matter what advanced driving systems you compare

It doesn’t matter? Sorry, but this is absurd. There are no other advanced driving systems on any production vehicles that is comparable to what Tesla is doing.

This kind of unbacked, vague, hand-wavey, arm-chair cynicism holds society back from making real progress. No thanks.

True, Tesla's ADAS systems are unique in crashing into things like big rigs, police cars, parked cars, children, and other things that every other system is able to avoid.

Silly pro-Tesla FUD is setting back actual progress in ADAS systems by years.

The baby steps makes sense, but you gotta admit Phoenix has to be thr easiest metro area to test with: Little inclement weather, little terrain change, newer infrastructure, streets mostly laid out in an east-west, north-south grid with consistent naming and numbering.

Waymo and Cruise have also been operating in San Francisco for a while, which is quite the opposite.

Most of Waymo's hours are in simulation anyway, and that can be as hard as they want it to be. Waymo's meatspace test track is also nothing but edge cases.

I understand that Tesla also learns in simulation.

simulation -> production

Hard mode would be anywhere that has snow.

That's probably the hardest problem. But I think you severely underestimate just how hard getting it to work in a place like SF (in rain and fog) is. To get to where they are is a serious accomplishment.

Tesla approach is to grief their customers into paying for feature they have no concrete plans to ever deliver on (in the absence of lidar/radar sensors it’s a crapshoot) -> profit. Worked pretty well

> Collect ten billion miles of data from guys driving down 280 -> ??? -> profit.

It's actually

Profit -> Collect ten billion miles of data from guys driving down 280 -> ???

Yes why build a technology when you can sell just the promise of that technology to your stans.

You don't think Tesla is building out their technology?

Not only that. Tesla thought they could do it (better!) with an inferior sensor suite.

The argument was more that to do it generally you need vision to work, lidar will always be a narrow solution and redundant when you have vision.

It might end up that solving the narrow city problem with high res maps is valuable enough to tackle anyway, but it could still be true that will never solve the general case.

Of course, it could be true that Tesla was be unable to solve it also. Though at some point attaching some general intelligence to vision can probably do it. Likely will be bigger issues then though.

Just because humans can generally do it without lidar and other sensors, does not mean that lidar and other sensors, would not improve human driving, if we could somehow integrate them seamlessly (and well reduce their price).

A car equipped with more sensors, will always be better at handling more situations, than the one without them, no matter how "smart" their drivers become.

I think the overlooked part is "seamlessly integrate". Signal to noise ratio is important, and it is not trivial to "just add sensors". I think the analogy of "too many cooks in the kitchen" applies here.

Indeed. Ultimately humans do it remarkably well with just vision, too, when they’re not intoxicated, distracted, or falling asleep.

Yeah, adding sensors can sometimes make things worse.

Example? There are plenty of examples where mistakes that Tesla's cameras make (being blinded by the sun, having trouble with emergency vehicles) are no problem for systems with more sensors.

This talk was worth watching (any Karpathy or Carmack talk is imo): https://www.youtube.com/watch?v=g6bOwQdCJrc

Specifically this is the relevant part: https://youtu.be/g6bOwQdCJrc?t=1370

They removed radar because it was making things worse.

I interpreted that statement by Karpathy as an accidental admission of Tesla's own limitations and not of the approach itself. (Note: I'm upvoting all your comments because they're very reasonable and supported.)

I think the keyword you’re looking for is ‘sensor fusion’, which apparently is only a problem for Tesla because the others have figured it out years ago. Perhaps there is a talent gap at Tesla or an unwillingness to invest in it. They generally avoid having to do anything that’s hard and try to look for shortcuts.

Case in point: the whole radar removal and reintroduction flip flop. It seems like they don’t believe themselves that cameras are sufficient.

I don't think anyone has "figured it out" if they still don't have unconstrained driverless cars.

That’s tangential to your original point of discussion which was sensor integration. Sensor fusion is absolutely a solved problem. Autonomous driving requires more than just sensor fusion though.

It's not tangential at all. If the purpose of these sensors is for autonomous driving, and "solving" sensor fusion hasn't gotten you there, how can you possibly say it is a solved problem? Sensor fusion (in this context) is not solved until you are using fused sensors for autonomous driving.

Why are you equating sensor fusion with solving the entirety of autonomous driving? That’s disingenuous. You understand sub problems can be solved, right? Sensor fusion is just combining different sensor inputs so you have high confidence in your object detection and you’re not “confused with disagreements” as Tesla likes to say. Self driving involves solving prediction and planning problems too, not just sensor fusion.

I'm not equating it with solving the entire problem, I am pushing back against the idea that autonomous driving can be sub-divided into completely discrete sub-problems. So I guess you could say no, I do not understand that sub-problems can be solved.

Object detection is inextricably linked with prediction which are both inextricably linked with planning.

Of course, they are linked. It doesn't mean you need to operate worldwide to prove sensor fusion is working. There is no scaling issue with it. Sensor fusion is already proven effective as companies using it are showing excellent safety records where they are operate. The Tesla complaints about "what do you do in case of disagreements?" don't hold good anymore. They just don't want to invest in get it working (but are now forced to as they re-introduce radar in their vehicles).

What evidence do you have that there is no scaling problem with sensor fusion?

As you point out, the companies that have "solved" it don't have scale.

Sensor fusion has nothing to do with scale. They are merely algorithms that combine multiple real time inputs into a single low level representation. This process remains the same whether you operate in San Francisco or worldwide. How they are interpreted is what differs from a Waymo to a Cruise to a Tesla. Waymo/Cruise get richer input they can feed into their downstream perception algorithms. Look at Waymo’s object detection benchmarks against Argoverse, KITTI and Waymo Open Dataset for their performance. They top most of them.

They don’t have scale yet for different reasons (adverse weather driving, long testing cycles, costly operations setup). “If they don’t have scale, that must mean sensor fusion doesn’t work” is quite a ridiculous argument.

If you think sensor fusion is not possible, then you must also believe Tesla’s attempts at re-integrating radar will not work and that they won’t achieve their stated goal of FSD.

I never said I don't think sensor fusion is possible. My only argument has always been "adding more sensors does not mean you will have a better self-driving car". Richer data does not mean better data. It does not mean more actionable data either. All it means is that you have more data points. I absolutely agree that this can be a good thing, and probably is in the case of self-driving.

But maybe it's not. Maybe the additional flops required for sensor fusion would be more useful if given over to the planning part of the problem. After all, you don't have infinite cycles. This is what I mean when I say it has not been proven at scale. The edge cases matter (arguably they are all that really matter), and "solutions with more sensors work better at object detection in artificial benchmarks" is not nearly as convincing to me as it seems to be to you. Again, object detection does not exist in a vacuum, and a solution that generally behaves better in a very constrained environment might actually behave worse when you throw all the edge cases at it and add on prediction and planning.

> Richer data does not mean better data. It does not mean more actionable data either.

How actionable you make it depends on how good your engineers are. Diverse input data is definitely more actionable than just RGB pixels from images.

> But maybe it's not. Maybe the additional flops required for sensor fusion would be more useful if given over to the planning part of the problem. After all, you don't have infinite cycles.

On the contrary, it reduces your compute requirements. For example, adding lidar means you are getting direct distance measurements instead of wasting compute cycles predicting distance using voxels. It's a much more effective use of compute.

You are getting direct distance measurements for some things, but not for everything, so you still need a system that is predicting distance using voxels.

LIDAR doesn't work great in the rain/snow, so you still need a system that is predicting distance using voxels.

Doesn't really sound like less compute to me. I feel like I'm repeating myself but again this is what I mean by scale matters. A self-driving system that performs better in perfect conditions is not even a good system if it doesn't work in the rain.

Which is the point the detractors of "fuse all the things" have been making. Don't get me wrong, I think probably those same detractors (e.g. Elon Musk) have also not been entirely intellectually honest with their arguments and have ulterior motives like cost savings and saving face, but that doesn't make the arguments wrong.

> A car equipped with more sensors, will always be better at handling more situations

Yes, integrating seamlessly is the hard part.

For Tesla, they found the increased sensor fusion complexity made the overall system less reliable, which was what informed their attempt at vision only. Karpathy went into this on Lex Fridman's podcast.

Only with better sensor fusion algorithms that worst case the overall system performance doesn't degrade with additional sensors. Tesla could have hit compute bounds in order to meet latency requirements for example (or they couldn't find the right technique/algorithms).

Sounds like a hard problem but I imagine this would eventually be overcome in the future. More sensors is definitely the future.

This is retroactive justification for using low-cost sensors. There's a chance they get lucky and pull it off with only cameras, but the chance of Tesla doing that is much less likely than Waymo getting their sooner with more sensors (or even even the Waymo driver achieving it with only vision if they chose. Using a full suite of sensors gives Waymo much better training data if they ever chose to use vision-only.

I think Tesla does use Lidar in training for what it's worth (at least I recall someone telling me this - I think it was a friend that worked there but not 100% sure I'm remembering correctly).

It's possible the lidar approach only really works with up to date high resolution maps and without that you end up in a local max you can't really escape with that approach.

That's been Tesla's argument up to this point anyway and I don't think the success in cities really proves things either way.

It may not matter since robotaxis in cities is still worth a lot and highway driving is mostly a solved problem.

True full self driving though will probably require solving vision - in that case I think the argument that lidar is a local max could very well be true.

> It's possible the lidar approach only really works with up to date high resolution maps and without that you end up in a local max you can't really escape with that approach.

Sorry, but you’re just repeating Elon Musk’s buzzwords like “local maximum”. It doesn’t make sense. There’s no such thing as “lidar approach”. All sensor inputs are fused (early/late fusion) and run through perception algorithms. Lidar is used for localization with maps, but that’s not even its primary use (that’s object detection). And it does not require maps to be up to date either.

I expect Tesla's rationalization will suddenly change when the cost for lidar units drops to hundreds of dollars. At that point, Waymo will have a huge head start.

Lidar cost has already dropped a lot. Waymo’s lidar on their I-Pace is 90% cheaper than the previous gen. We’re also seeing car manufacturers (GM, Volvo/Polestar, Mercedes, Chinese OEMs) include inexpensive lidar units for driver assistance.

Tesla’s problem is that they’ve promised their existing cars on the road are capable of full self driving. They can’t add hardware to them, so they will maintain the “camera is sufficient” stance for a long time.

Birds fly by eating bugs and flapping wings, but that doesn't mean we should build planes that way. Tesla is stubbornly handicapping itself by cheaping-out on sensors because they think they can hail-mary yolo themselves all the way full self driving.

The Tesla's do have more than two cameras despite us having only two eyes. I don't think that analogy totally holds. It's more that because roads (and road signs) are designed for human vision and human drivers in order to handle all cases you really need to solve vision and if you do that sufficiently then the other sensors are redundant anyway.

High resolution mapping works for localized solutions, but not in the general case.

I'm glad both groups are working on different approaches, it'll be better for us in the long run imo.

High resolution maps are only one part of the equation. Everyone is working on a generic perception/prediction/planning stack that works everywhere. Only the map changes between different cities. This is how Waymo/Cruise have been able to deploy in different geographies.

Proven in what way? As a way to lose money?

It's no surprise that it's easier to get to Level 4 if you have a 50-100k sensor suite, HD maps and a geofence.

However, I wouldn't be shocked if, even if Tesla robotaxis arrive many years later, Tesla achieves profitability before Waymo does.

Tesla's FSD had been very profitable for a long time already. To most of us that is not as impressive as operating actually robotaxis at whatever cost

Last based on what metrics? I've never seen anything better than the Tesla FSD except for the LIDAR robo taxis, but they are solving a very different problem with different constraints. Most recent cost estimate I can find shows a cost per vehicle of $130-150K for Waymo vs the Tesla system that needs to go in a vehicle costing $40-50K which makes using a Waymo style sensor suite impossible. Would be interesting to see how the Tesla software would perform with the sensor input of something like the superior Waymo suite, but we won't ever find out unless costs come down by a massive amount.

Your numbers are very outdated. Exact figures and targets aren't usually announced, but both Waymo has talked about their upcoming vehicle being designed around a cost/mile ratio lower than consumer vehicles. That's not possible without very reasonable production costs. Cruise has also announced a target production price for their vehicle of $50k at scale, which is around the price of a new Tesla. That's not apples to apples, but it's close enough.

Tesla announced a 25k car. All of the numbers so far from every company are just speculation, though. We’ll see how things actually pan out.

Sensors are following a really great learning curve, rapidly getting both more capable, more reliable, and cheaper. It's reasonable to expect they could be a few hundred dollars per unit soon.

What does “running the driver in hard mode as much as possible” mean?

Why do you think Tesla’s approach “didn’t work and they are now dead last in the game?”

Maybe that’s just an opinion because you provided nothing to back it up - but I can tell you not only does AutoPilot/FSD Beta work, the latest FSD Beta stack (I’m driving 11.3.6) works extremely well.

Tesla has orders of magnitude more passenger-miles driven than any other solution, which is one quantitative way to gauge success.

Tesla’s latest FSD Beta safety record on city streets is also excellent - 10x fewer accidents per mile than Cruise for example.

This is another huge win for Tesla but it’s important to note that this is accomplished in part by requiring strict driver oversight (enforced through gaze & attention tracking in the cabin camera). So the significantly lower accident rate is not an apple-to-apples comparison of two L4 solutions, but it convincingly shows that their L2 solution is actively making the city and highway roads safer for the driver and the cars & pedestrians that they are sharing the road with.

Reply on news.ycombinator.com