Hacker News Reader

Comment by RIMR | Hacker News Reader

> But after the study controls for age the starkness of the effect more or less disappears (eyeballing Fig 1, the 2nd graph).

Adjusting for age, Alzheimers deaths were 1/100 for taxi drivers, and 1/60 for the general public. That's extremely significant. However you feel about the graphs means nothing compared to the actual numbers presented here.

> I'm sure it is a great study. Lots of science done. They do good graphs and all that.

Your choice to be dismissively anti-science helps to explain the bad point you're making.

View on HN

Comparing 1/100 and 1/60 doesn't mean anything in terms of statistical significance (you would need to know more).

They aren't being anti-science and it's low-quality to suggest they are.

> That's extremely significant.

What makes you think that is significant at all? Anytime you take a lot of small subsets of a large random sample you expect to find outliers that look a lot like this.

That is https://xkcd.com/882/ seems to better fix the data just as well.

Outliers, sure. Exactly like this though? That depends. Since this is a population then assuming a gaussian (which could well be wrong for a variety of reasons I'm just guessing here) what's sigma? If 1/100 is within ~3 sigma then sure. Otherwise no.

It needs more statistical analysis than I have the background to do. However all signs look to another variation of the replication crisis, which is more an more my default assumption when I see an observational study.

You're handwaving. The xkcd point only applies if you're within ~3 sigma. This isn't a complicated statistical analysis it's just asking how far out the outlier is versus the standard deviation and size of the dataset.

To be clear I'm not claiming in one direction or the other. I feel that if you're going to attempt to discredit a piece of work it's on you to do such a basic check.

Really the whole point of that xkcd was to viscerally illustrate the practical effect of binary misclassification, or alternatively to make the point that a p of 0.05 isn't necessarily as rigorous as you might expect. Neither of those things directly applies here.

The article doesn't give enough information to answer your question. It isn't on me it is on the reporter's who shouldn't allow a handwave like that to pass without question. There are far to many cases of people hacking statistics in the world to accept that.

It is an outcome from over 400 data points. We're expecting to see events outside a 3 sigma boundary. In fact, 3 sigma is basically the 1:400 boundary. Strictly it is 1 in 370 [0] for anyone who takes Wikipedia at face value.

In the study they seem put the P value of the taxi driver outcome at <0.01 (in Table 2, although I admit I haven't read the thing especially closely). For 400 data points, we expect there to be around 4 values with a P value less than <0.01.

[0] https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rul...

Reply on news.ycombinator.com