Hacker News Reader

Comment by nonbel | Hacker News Reader

Ok, but please do not vote for anyone calling themselves "scientist" who practices NHST, you will only be perpetuating this nation- (even civilization-) destroying practice:

"We are quite in danger of sending highly trained and highly intelligent young men out into the world with tables of erroneous numbers under their arms, and with a dense fog in the place where their brains ought to be. In this century, of course, they will be working on guided missiles and advising the medical profession on the control of disease, and there is no limit to the extent to which they could impede every sort of national effort."

Fisher, R N (1958). "The Nature of Probability". Centennial Review. 2: 261–274. http://www.york.ac.uk/depts/maths/histstat/fisher272.pdf

Do not be blinded by a degree, there are a lot of these people out there. As a heuristic, you may have to limit "scientist" to mean physicists and engineers, as those communities have been least affected.

View on HN

>Ok, but please do not vote for anyone calling themselves "scientist" who practices NHST, you will only be perpetuating this nation- (even civilization-) destroying practice:

Can you expand on this? What is NHST?

Null-hypothesis statistical testing

The parent poster has ... for some inexplicable reason ... decided that this technique is invalid. This reeks of a profound misunderstanding of the technique, and a failure to discriminate between poor use and invalidity.

It isn't inexplicable. NHST entails affirming the consequent and transposing the conditional, which are known formal fallacies.

Null hypothesis significance testing. It is a "method" of statistics that resulted from confusion between Ronald Fisher's significance testing with Jerzy Neyman and Egon Pearson's hypothesis testing sometime in the 1940s. None of those three are free of blame either.

While there is all sorts of things wrong with NHST, the primary thing is that researchers never actually test their own hypothesis. Instead they test a "null" hypothesis that acts as a strawman, usually that some parameter equals exactly zero. Then when the strawman is "rejected", they go on to say the results support whatever theory/idea it is they favor.

Rather than come up with precise predictions deduced from their idea, they leave them vague (If I am right, A will be correlated with B). Then calculations are used to reject the very precise strawman, giving an entirely false sense of rigor to a handwavy, nonsensical ritual.

It is very common, and seems to get more and more popular every year despite all efforts by scientists and statisticians to stop it (because it lets you seem scientific about proving whatever you want). I was trained to think that way myself not too long ago, with absolutely zero mention of any controversy.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4347431/

http://library.mpib-berlin.mpg.de/ft/gg/GG_Mindless_2004.pdf

http://www.fisme.science.uu.nl/staff/christianb/downloads/me...

http://andrewgelman.com/2016/09/30/why-the-garden-of-forking...

>Instead they test a "null" hypothesis that acts as a strawman, usually that some parameter equals exactly zero

How is "there is no difference between my groups/conditions" a strawman?

This is by far the most common H0.

Because, except in rare cases, the model/theory/idea of interest does not predict "there is no difference between my groups/conditions". It predicts something else. In the rare cases a zero difference is predicted, the method is scientific.

For nearly all use cases many plausible models/theories/ideas will predict "not exactly zero difference", so rejection of the hypothesis "difference = zero" is useless to the scientist. Mathematicians apparently don't get this, which is what inspired the Fisher paper I linked to.

And yes, it is the most common H0. No argument there. The problem is so widespread and destructive that the mind instinctively rejects its existence as unbelievable.

Sorry, I still don't understand. Why would an absence of effect predict anything other than a zero difference (assuming infinite samples)?

No one predicts "an absence of effect". Don't worry, it took me years of reading the literature, running my own simulations, and having discussions in person and online. It is really difficult to accept how bad this is.

Please stop being condescending and actually explain yourself.

>No one predicts "an absence of effect"

In my field, we most certainly do predict an absence of an effect, in the following sense:

1. if a manipulation does not have an effect on the system, then we expect no systematic deviations in the measurement distribution

2. point 1 will be revealed by H0 (no systematic change in measurement distribution across conditions) not being rejected.

It's beginning to sound like you've encountered people who misuse or misinterpret this technique -- and I agree that there are alarmingly many such people -- but this is different from the technique being invalid.

Are you suggesting that use (or otherwise) of statistical significance tests in a scientific field would have an affect on your performance as an elected representative?

First, there is a difference between a significance test and a null hypothesis significance test. The former was proposed by Ronald Fisher, the latter is something different that arose anonymously. See my other post for some links.

I'm suggesting that people who do NHST would encourage more funding to go towards others who do NHST, while also making policy decisions based off of it. At least when politicians are mostly clueless about science they use intuition and common sense.

Reply on news.ycombinator.com