Even after $100B, self-driving cars are going nowhere

Are you on Beta 10.69.2.3?

If I had to articulate my reasons:

* Judging by the videos available online, my perception is that many situations that were impossible for Tesla FSD Beta a year ago have become uneventful in recent weeks. Take a look at Chuck Cook's videos for example (I like the fact that he always highlights the failures).

* Judging again by the videos available online, my perception is that Tesla FSD Beta has encountered and had to deal with more crazy edge cases than any other system. A possible explanation for this is that for a long time Tesla FSD Beta hasn't been geofenced or restricted only to certain types of roads, like highways. You can test it anywhere in North America.

* Tesla FSD Beta currently has 160,000 individuals testing it without road restrictions. As far as I know, no other system has been exposed to similar open-ended large-scale testing.

* Occupancy networks look like a real breakthrough to me -- DNNs that predict whether each voxel in a 3D model is occupied by an object, using only video data as an input. I understood the high-level explanation of these DNNs on AI Day 2. I haven't seen anything like it from anyone else.

* Tesla's DOJO also looks like a breakthrough to me. I understood the high-level explanation of it on AI Day 2. IIRC, DOJO cabinets are 6x faster at training existing neural networks than Nvidia rigs, at 6x lower cost, so call it ~36x more efficient.

It’s a completely different problem space, like claiming someone built a train and therefore they can easily build self driving cars since they are both “driverless”.

I don't think that's the actual reason for the decline, it's more likely another effect of the real cause. While I'm not American, it's the same story all over the world: our diet keeps getting worse, there is plastic everywhere which breaks down into micro plastics which end up in the water we drink etc. We're preparing our food with carcinogenic utensils (everything "non stick") and all of our western societies went from a healthy worker/academic career choice into mostly just getting exploited by the established players.

There have been a lot of societal changes since the post WW2 times, and summing that up into "losing trust in medical science" is a very confusing take, especially if you consider all the medical professionals that spread outright lies to profit. (The person that started the antivax movement was a doctor as a notorious example)

And drug companies that sold bad medicines (knowingly!) As they judged the cost of liability to be lower then the cost of not selling the drug.

Just to be clear, I'm not saying that any of these things are causing this decline either. Our societies have just changed too much since to make any confident claims about their impact.

Part of this is the fault of Tesla's marketing, but you are wildly off mark. The cars you are seeing are Autopilot, not FSD. Most of them are even the even older, radar-based autopilot.

Tesla Vision has no issues detecting white semis crossing your path. Vehicles with radar, on the other hand, struggle with discerning those from overhead bridges, so if one appears close to a bridge, you're SOL due to whitelisting.

Tesla Vision in FSD is a much, much more developed version which has been excellent about detecting its environment, especially now with the new occupancy network. Its decision making needs work but you will notice, when watching all those videos, that detection of vehicles - even occluded ones - is not a problem at all.

Your comment about useless data is also wrong. They are experts in their field and they know exactly what type of data they need. Both Tesla and Karpathy himself have shown on multiple presentations that they focus on training unique/difficult situations because more data from perfect conditions is not useful to them anymore. They have shown exactly how they do it, and even showed the great infrastructure they've built for autolabeling.

Your claim about cruise control from competitors being equal to FSD is laughable. They don't even match Autopilot: https://www.youtube.com/watch?v=xK3NcHSH49Q&list=PLVa4b_Vn4g...

People tend to forget just how hard the edge cases in vision are!

It feels like Tesla’s main strategy is to add more data, more compute power, more simulation, and hope for “convergence”. Maybe that will work, but right now it feels like Cruise’s technology feels more mature and thought through.

That asset will probably be worth far more than the self driving system.

It seems like Tesla has a huge advantage in terms of training data by leveraging a fleet of millions of vehicles.

Which is more valuable, experience or technology?

I was driving down the road as normal, 4 lane divided highway that's a bit hilly. Suddenly my car starts having what I can only describe as a panic attack saying I'm running a stop sign and blaring alarms.

It was detecting a giant 40ft tall red circle sign a bit away as a stop sign...

Between one and two years ago, that was my perception too. But the rapid progress I've seen with Tesla FSD Beta over the past couple of years, and over the last year in particular, has forced me to change my mind. (Note: I'm talking only about Tesla's beta software. Tesla's production software is behind by dozens of versions and is without a doubt technologically inferior to Waymo and Cruise.)

Less than two years ago, I would have said FSD Beta could only deal with the "first 95%" too. Now, my perception is that FSD Beta routinely handles the first > 99% and fails only on < 1% of situations. Moreover, the failures have become more graceful -- e.g., the car will stop at intersections perceived as risky and ask the driver to confirm go-ahead by pressing the accelerator. If FSD Beta continues to improve, sooner or later it will cross the threshold at which it becomes safer than most human drivers.

Of course, IF I see new evidence that contradicts my perceptions, I'll change my mind again. There's no shame in changing our minds when the facts disagree with our views. FWIW, I'd love to see videos of Cruise and Waymo vehicles, filmed by tens of thousands of regular consumers driving autonomously on fully unrestricted roads, with zero editorial input from Cruise or Waymo.

Different approaches lead to different paths to solutions, I am not convinced that either will be successful and not convinces Waymo/Cruise is ahead.

Unlike those, Tesla actually makes money and uses the technology stack in more limited forms.

Do people who work in this industry actually think they've solved 95% of driving scenarios because their software can manage driving in sunny California, Nevada, and Arizona?

My brother lives in the countryside (somewhere in the EU), and, as a driver, he shares the paved road just outside his house with the village’s cows (including his two cows). I don’t see any non-AGI system being able to negotiate that, as at times is difficult even for me, a reasonably AGI system, to make sense of it all when I encounter a herd of loose cows on the road.

And this is with stationary objects designed to be seen and easily grasped by humans.

This is -not- my line of work, so I have no idea if that's true. But if it is, I don't see how we could have perfectly safe self driving vehicles.

Humans are good at predicting.

You don't consciously know you've seen the guy a couple of cars in front checking his mirror and his shoulder but you're hanging back because you just know he's going to pull out any second. The guy that's wavering a bit in the middle lane is about to dash across to the far lane of the sliproad that's coming up, clipping the zebra stripes a bit, because he's concentrating on the sat nav not the road, but you just know - out of all the other drivers in your space at the moment - that red Ford is the one that's going to do something boneheaded.

Autonomous Vehicles won't be able to do that, probably not ever.

Everyone is committed to being green as long as their lifestyle isn’t inconvenienced and they have the funds to buy the green equivalent technology.

It's also easier to avoid pedestrians as most can't fly.

It also could save on gas and energy as you can go directly as a bird flies to your destination instead of taking 20 minutes it takes 3.

Of course this would be a huge infrastructure ordeal as well probably and require a damn good system as you don't want vehicles landing on houses all over the place, but it'd be amazing for people with long commutes.

You could probably have airbuses that pick up like 30 people say in a small town and fly them to the city to be delivered individually by smaller vehicles locally. What took 45 minutes, now maybe takes 15.

These would be better if maybe electric with gas as an emergency backup system, and then just have good batteries and solar power fuel most trips.

Sounds somewhat authoritarian to impose your idea of the 'future' to inconvenience a large number of people.

American corporations perhaps aren’t as interested in public transportation, because there is no money to be made. And that is who is largely funding this self-driving vehicle research.

I only ever take a car for trips outside the city.

There are lots of places all over the world where $1B would make a big difference in many, many people's lives -- and where people eagerly ride the busses they actually have, which are often not that nice.

Do we need things to work in the USA for them to be worth doing?

If you're still stuck with peak road usage nearly equivalent to that today, your goal isn't to solve prominent issues today.

Great so you have constantly cars driving form the city center to outside of the city, that for sure will cause no traffic at all.

Will be fun when people proposes new elevated highways out of the city so the self driving car can go outside of town to the coal power plant to charge.

The only thing worse then having vehicles driving around with 1-preson, is vehicles with 0-people. Its literally the most inefficient use of space ever.

It makes traffic worse, not better and it makes the city worse, not better.

How about this, a city optimized for walking and biking, where different parts of the city are connected threw buses, trams, subways or regional trains.

> Especially, if the existing, personally owned cars that just stand around 99% of the time vanish over time.

Turns out that cities where people can, walk, bike and take trains they don't own cars. Shocking.

Its not about 'snapping your finger'. Neither Netherlands or Switzerland built their systems from 1 day to the other.

You need to make decision to change and then consistently and incrementally work on it. Put it in your standards and invest ever $ you have for new roads to that instead.

You need to change your tax policy so that horrible inefficient land uses like parking lots cost a lot more. You need to enable mixed use development so these parking lots can be built on.

> To do public transport right, you'd have to basically demolish the entire city and re-build everything from scratch to be friendly to pedestrians.

I'm sorry that is complete and utter nonsense. Like seriously, completely insane.

If you look into some urbanist and city planning literature you will see that lots of places where there used to be total car shitshows, are now beautiful. Often you would never have guessed that just 10-20 years earlier it was horrible road and a parking lot.

Again, small and incremental steps. Here are some really basic steps you can take:

- Remove parking requirements

- Slow speed of cars

- Don't allow turn right on red

- Make the lanes thinner

- Make the sidewalk broader, maybe add some trees

- Take one of the existing lanes and add painted bike lanes, later add protection for those lanes

- Rezone for mixed use (specially existing commercial zones)

- Change property tax policy to discourage sub-optimal land use

I could literally keep going on and on. Non of this, requires you to demolish anything.

Specifically for the US, there is whole movement about incrementally improving your city, see Strong Towns (https://www.strongtowns.org/). They have lots of podcasts and books. Specially: 'Strong Towns: A Bottom-Up Revolution to Rebuild American Prosperity'.

They also point out in detail with real data how these changes make your city safer and economically much better (They have some seriously amazing visualization of city finances that shows how such chances can improve cities).

And this is not some hippy organization, these are coming from a somewhat conservative small towns perspective.

Honestly your attitude of 'we are stuck with this' is horrible. I can understand frustration and bleak outlook, about the situation. But put your hope into incremental low cost change, not some techno futurism and you will be less disappointed.

What is that evidence, exactly? I agree that we might eventually get there, but the scale seems to be 50-100 years at this point. We are as arrogant as the researchers in the 60s who famously announced that absolutely perfect image recognition is only 1-2 years away - except the problem is several orders of magnitude harder.

However, there will inevitably be conditions that require the use of general intelligence (rather than driving heuristics), and in those situations all you can do is pray the computer acts rationally despite not having GI.

I think self driving cars have already passed the test of "number of crashes" or "number of fatalities" per mile driven. But I don't think that's enough to sway the public, if every once in a billion miles a self driving car slowly drives off a cliff for no apparent reason.

A lot of the self driving tech has already made its way into safety systems in cars. Things like automatic breaking seem generally useful. There's a question about if it's worth $100B for research into partially autonomous safety solutions, but I don't think it's useful to attribute zero value to self driving research until we get full self driving (certainly some value is being realized already).

Thats how I read it, sorry. We made tremendous leaps in the past decade with improving automated driving, is it fully automatic? No. Does it mean we should somehow stop funding it? No.

When a human drives a car, their only sensors are eyes, ears, and maybe vibration. Somehow we manage to muddle through it.

Why do L4/L5 cars need anything extra sensors-wise?

I repeat my comment on LiDAR that I gave a few days ago. The gist is that LiDAR is cheap and you will be able to buy a LiDAR with sufficient resolution for in the next 1-2 years because it will be integrated in normal passenger cars for L2/L3 assistants. These cars are coming out now or in the next year.

LiDAR is finally getting cheap. OEMs (like VW) are very price sensitive. It is estimated the sensors from Valeo cost about 500 dollars. The fact that you see more and more normal passenger cars with higher resolution LiDARs means that LiDARs are getting cheaper.

The Audi A8 used Valeo's (with Ibeo) first generation low resolution LiDAR Scala 1 from the automotive supplier Valeo. Mercedes new models will be using Valeo's second (or third) generation LiDAR. All these are used for L2/L3 assistants. Valeo is a traditional large automotive supplier.

Luminar, a public company from the US, cooperates with Volvo. Some models will come with a LiDAR in the base configuration. These are "new LiDARs" with high resolution.

Innoviz, a 'startup' from Isreal, will supplies LiDARs to VW. Its angular resolution is (in its focus area) about 0.1 (or 0.2) degrees, which is sufficient for higher levels of autonomy and surpasses/equals the resolution of the expensive Velodyne sensors of the past. They will probably be in the same price range. Due to the limited FOV due to the technology, you will need multiply LiDARs.

Many new models from Chinese car brands will also ve equipped with a LiDAR. Most of them with Chinese LiDAR manufacturers like RoboSense or Hesai. Some are equipped by European manufactures like Ibeo/ZF. For example, there is the automotive sensor AT128 by Hesai. It targets normals vehicles (see price range above) and claims a similar performance (except for FOV, so you need multiple) like the Velodyne Ultra Puck (~$50000).

So costs of LiDARs are a not the very expensive obstacle they were in the past. The only problem could be that the new LiDAR manufactures cannot scale up series production. For example, Ibeo just filed for insolvency because they could not close another round after aggressively increasing spending in the past years.

This is an important piece I have not seen addressed in the US. In places where there is snow and ice on the road a good part of the year self driving cars will need help from sensors and guide objects in some form. Perhaps sensors injected into the road? Humans barely manage in my area because people have a cognitive awareness and memory of the terrain. Road lines are often absent. Sidewalks are obscured. Even simple things like parking at the grocery store is relative parking and people just make a best-guess as to where a spot is.

I am also curious if any testing has been done in snow blizzards and squalls. Squalls can occur without warning and visibility drops to nothing.

I can think of many reasons why train wouldn’t be an option.

Furthermore, all of those research prototypes are geofenced, since they rely on extremely detailed mapping and lots of training data from humans driving the same routes. There is not a single AV in the world close to capable of driving on any road, at any time, in any condition, that a skilled human could drive on without ever having seen it before.

In my mind, that means we are indeed nowhere close to a true AV.

Obviously we don't know to what extent they have remote drivers but you wouldn't be able to run such a service if every ride required them.

[1] https://www.youtube.com/channel/UCP1rvCYiruh4SDHyPqcxlJw

Take the driver out of the equation, and suddenly making the train longer doesn’t provide any savings. You might as well switch to smaller and cheaper engines and divide one train into 10.

There are a lot of variables that affect efficiency but new technology alters the landscape. Self driving public transit is one such thing. Would you rather be on a bus with one driver or a self driving taxi that costs the same?

Rather than assume this is a death knell for self-driving cars, a more pragmatic view would be other nations will be able to benefit from them while you are left behind. Self-driving cars don't need global saturation to be worthwhile.

I do think in some areas this will not be feasible. But these are also probably the very last areas to consider when building smart car infrastructure.

Because you guys and your parents chose to make you dependent on that. The solution is not to build more killing machines and spend more time behind the wheel. All that has been done in the second half of the century can be made differently.

An uber driver that can drive 24/7. Also there are a lot of cars that sit in parking lots 95% of the day that can be used.

I think that this is a sunk-cost fallacy. Sure, you would need to build infrastructure for trains, but after that trains are now reasonably automated, can carry cargo, and do other things with the exception of stopping at the exact destination you wanted to (in other words, it requires predetermined stops which people outside the US shrugs and just walk or bike the last feet). Also, they're proven to work: even China (which previously didn't have trains to its far-flung places) and they've done what you've expect. I think that the sole reason that anyone wants to invest in automated (non-train) driving is because trains are boring while AI is oh-so-shiny.

(although the risk is clearly that vehicles will transition into subscription services that cost many thousands of dollars a year and price them all out)

You don't need full autonomy to mostly fix this, but the fixes are politically untenable (currently). The cars should be speed-governed, they should be speed-limit aware -- the car should routinely be overriding the desire of the driver. No, you can't go into the bike lane to get around traffic. No, you can't make the split-second decision to swerve around a car braking in front of you. No, you can't operate the vehicle at 100 mph in a residential zone. No, you can't go that fast right now; it's raining way too hard, doofus. No, you can't operate your vehicle onto a scheduled parade route.

You don't need full autonomy to create cars that prevent a significant percentage of driver -- to put it charitably -- errors. These are fairly straightforward problems. The opposition is political.

I think this attitude is quite common, but people won’t admit it. I would never admit it like I just did if I didn’t have an anonymous account. And I don’t see how technology will change this basic preference.

Could is the question, but should is never asked or answered, same goes to other projects like solar roofs with the shingles.