Hacker News Reader

Comment by burningion | Hacker News Reader

I think everyone using this product should be asking themselves why we don't have an objective measure for the quality (or lack thereof!) of the product.

With every new release of FSD, there are people quick to say it's a breakthrough, and amazing, while others lament it doesn't nearly do what it says on the lid.

Why are we putting safety critical systems we can’t adequately measure on our streets?! There are people who confidently state that Tesla's FSD is already better than humans.

Where's the data?! Why isn't it open? Why can't we build an objective measure for effectiveness and safety of each of these releases?

View on HN

The data isn't open because it's not required to be. Every manufacturer is either making a good faith effort to collect that data (to their own standards) or they probably shouldn't be on the road. Regulators, NHTSA especially, don't have the expertise, time, or mandate to get it from them, analyze it, and make it available. That could (and should) change though.

As an unrelated aside, measuring safety performance is hard. Doing it accurately without putting vehicles on the street is probably impossible. Simulation is not a fully adequate replacement for road testing and many companies are already looking at cutting the latter because of the expense. It'd be nice for regulators to outline an acceptable road test framework that better balances the goal of public safety from unsafe vehicles against ensuring safety can be demonstrated in real world environments.

Actually, there are some attempt to measure it.

https://www.dmv.ca.gov/portal/vehicle-industry-services/auto...

Of course Tesla is not participating - the real numbers would make them look disastrous.

Regarding objective measures for the quality - the famous standards for level 1-5 etc. Objectively speaking, Tesla's system is Level 2. There are other companies at level 3, and some others operating fully autonomous vehicles.

As to why Tesla chooses to base all their claims on purely subjective and highly cherry-picked numbers and videos ? We all know the answer to this question.

> Tesla's system is Level 2

Which I'm happy with. I do 95% of driving on Autopilot already. Add another 4% for FSD that I'm happy to babysit and I'd be very happy.

For some reason people are so caught up on naming choice and what it implies...

There are several reasons - for some it's about security, for other about truth.

I find it fascinating that a guy become the richest person in the world, by convincing 'investors' that a Level 2 system is 'almost ready' for a Robotaxi operation and will make them infinite money. To give him credit, he also did mention that Tesla is worthless if it doesn't solve FSD.

Now this is an opinion I can get behind.

I'd love to see all cars (including Waymo) compete on a standard set of tests with clear gates as to what conditions they'd be allowed to go to Levels 3, 4 or 5.

Reply on news.ycombinator.com