Hacker News Reader

Comment by lordnacho | Hacker News Reader

Wow, adjusted for driver age the difference vanishes.

That's quite an interesting observation, and it shows the importance of proper statistics. It also raises questions about companies producing their own stats, it's often only when someone takes the time to dig into it that we discover this kind of thing.

I am not holding my breath waiting for the common consumer to understand statistical critical thinking though. I sometimes catch myself forgetting the due diligence even after many years of forcing myself to think about stats.

Perhaps some sort of ombudsman could do this, pointing out when stats are lying to people.

View on HN

> It also raises questions about companies producing their own stats

Companies saying "trust us, we have the data to prove it" while keeping that data secret should be ignored. Especially if their CEO has a long history of blatant dishonesty! I'm most annoyed that people gave any weight to Tesla's safety statistics claims in the first place.

> Companies saying "trust us, we have the data to prove it" while keeping that data secret should be ignored.

While true, I do not think this is a great heuristic. How would the average person know whether the "data" is available? Most people don't even know what "data" really is. They have never heard of CSVs or SQL. We need a better signal for consumers to know what they can and can't trust.

An average person doesn't need to directly access the data. As long as journalists or researchers can, an avg person could rely on their analysis/opinion instead.

I would certainly not rely on journalists' to actually have a look at the data, let alone to have the technical skills to analyze and report on it. As it is, reporters don't bother mentioning / linking to the data, or mentioning the lack of publicly available data.

It's OK to expect better from both journalists and companies.

Data-journalism is actually a thing, and bigger organizations like the New York Times have done some pretty impressive work building tools and training programs for their reporters.

https://open.nytimes.com/how-we-helped-our-reporters-learn-t...

Here we are in the comment thread of an article of a journalist reporting this very analysis.

A good journalist will include that context in a story. I know I read at least one article that included these safety claims while also noting that the data was private and unverifiable.

I think it's a good default to assume any self-beneficial corporate message is false, absent evidence to the contrary.

Companies dont validate their claims, only provide data and statistics to back up marketing angles. There's no point to it either since it would still be a conflict of interest when you've researched your own product and found no errors in it.

> I'm most annoyed that people gave any weight to Tesla's safety statistics claims in the first place.

TSLA stock consistently made gains for a lot of people who otherwise wouldn't be paying attention let alone putting their weight in anything Tesla related.

Driver age isn't the factor that explains the difference, accident rate adjusted for age shifts up a similar amount for both Autopilot and regular drivers.

What truly makes a difference is the kind of road they're driving on. The accompanying paper provides a figure showing Autopilot is essentially only used on freeways which are inherently safer. Adjusting for that seems to explain most of the difference.

Though personally I'd prefer to have just compared the accident rate on freeways for both types rather than this weird weighting method they try to use.

I highly recommend How to Lie with Statistics (1954) to learn more about these sorts of misleading stats: https://en.wikipedia.org/wiki/How_to_Lie_with_Statistics

If you have taken a course in statistics you'll know its always possible to make the numbers looks favorable to your cause. It would take several unrelated blind third parties to do this correctly and no public company will ever let that happen, especially Tesla.

They should be forced to let that happen. It's crucial for public safety.

Public knowledge of safety affects stock price and brand recognition. Will never happen.

This is an arrogant amount of speculation based on nothing more than personal intuition.

In fact, there are plenty of examples where public knowledge of safety is published by mandate of the government. Here is an example of FAA flight data and you can see details about incidents across airlines: https://www.faa.gov/data_research/accident_incident/

I'm not going to spend time digging it up, but I'm pretty sure there are just as many sources of data on other important things such as pharmaceuticals e.g. the covid vaccine.

So no, just because 'stock price go down' doesn't mean a thing will never happen

You could easily have a Simpsons paradox, e.g., it's safer overall, but for any single age group its neutral or even less safe.

This isn't "proper statistics", this is P-hacking. Tesla's data is incomplete and bad. But going out and finding confounding variables to confirm your priors isn't the treatment.

They should adjust for car-age and price.

Reply on news.ycombinator.com