Hacker News Reader

Comment by karlkatzke | Hacker News Reader

I'm not a statistician, but the results section is worth reading -- the fructan results are not much higher than placebo but both placebo and fructan are more affective than gluten.

"Overall GSRS-IBS scores differed significantly during gluten, fructan, and placebo challenges; mean values were 33.1±13.3, 38.6±12.3, and 34.3±13.9, respectively (P = .04). Mean scores for GSRS bloating were 9.3±3.5, 11.6±3.5, and 10.1±3.7, respectively, during the gluten, fructan, and placebo challenges (P = .004). The overall GSRS-IBS score for participants consuming fructans was significantly higher than for participants consuming gluten (P = .049), as was the GSRS bloating score (P = .003). Thirteen participants had the highest overall GSRS-IBS score after consuming gluten, 24 had the highest score after consuming fructan, and 22 had the highest score after consuming placebo. There was no difference in GSRS-IBS scores between gluten and placebo groups."

View on HN

This was my thoughts too. The values appear to be fairly similar for all three, especially given the large variations.

Not to mention that they're explicitly trialling this with people "for whom celiac disease had been excluded". So an alternate conclusion could be "people without celiac disease but with self reported gut issues don't respond significantly differently to fructan, gluten, and placebo". Which is... not surprising?

They don't appear that similar to me - if you plot them graphically you'll see a significant difference, particularly at the lower end.

P values of less than .005 generally mean that you accept the null hypothesis. The authors did otherwise for one hypothesis, which I find interesting.

You're wrong on two accounts.

You want a low p-value, the lower the p-value the stronger the evidence.

A p-value is cutoff traditionally at .05 (1 in 20), and if it's p < .05 you can reject the null hypothesis. The "null hypothesis" is the OPPOSITE of your paper's hypothesis (so you want to reject it): the null hypothesis is "All the variation we saw was from chance"

Why not make the null hypothesis the same as your paper's hypothesis? Wouldn't that make more sense?

The null hypothesis signifies a lack of relatedness. Give circumstance A and outcome B, we make no assumptions, i.e. we assume A and B are unrelated by not assuming they are related. This is the null hypothesis. In order to reject that, we must provide compelling evidence.

If you start by assuming A and B are related, your experiment would have to fail reject that notion in order to prove that A and B are indeed related, which is a less direct way of approaching scientific discovery.

You are also confused about the meaning of "null": https://news.ycombinator.com/item?id=15703503

No, they're not. The commonly accepted definition of Null Hypothesis is "the hypothesis of no effect or no relationship" (source: statistics master's degree, but it's also the first sentence in the wikipedia article: https://en.wikipedia.org/wiki/Null_hypothesis)

That is an awful wikipedia page. How can that even make sense? As soon as people start predicting things it is impossible to test the prediction?

Also, the intro of that page contains blatantly incorrect statements:

>"In the significance testing approach of Ronald Fisher, a null hypothesis is rejected if the observed data are significantly unlikely to have occurred if the null hypothesis were true. In this case the null hypothesis is rejected and an alternative hypothesis is accepted in its place."

There is no alternative hypothesis in Fisher's significance testing.

Here is a better wikipedia page:

"We may, however, choose any null hypothesis we please, provided it is exact."-Ronald Fisher https://en.wikipedia.org/wiki/The_Design_of_Experiments

I've heard that a lot of math fields really fail to teach history, perhaps that is what happened in your stats program.

You cannot prove the null hypothesis; you can only disprove it. The p-value is the probability that the null hypothesis is true. So if the null hypothesis is that there is no difference, and there is a low probability of that being true, then you have shown that there is a difference between the groups. If the p-value is high, you do not show that the null hypothesis is true. Instead, you show that you did not find a statistically significant difference. This can happen when the difference between the values is small or there is not enough data to make the difference clear, which is why a high p-value is not enough to reject the alternative hypothesis.

> The p-value is the probability that the null hypothesis is true.

No, the p-value is the probability that the given data generated is as far away or farther by random chance given the null hypothesis is true. That is to say, we assume the null hypothesis to make some predictions and see if the data is a likely occurrence under those assumptions.

This is not the same as the probability that the null hypothesis is true. If that is what you want (and most of us do want this), then Bayesian methods are more appropriate though they are more complicated and more sensitive to initial assumptions.

>"You cannot prove the null hypothesis; you can only disprove it."

Sure, but thats why the null hypothesis should be predicted by your theory. Then you are checking your theory. Most people could care less about whether there is a miniscule difference between groups or not.

>"The p-value is the probability that the null hypothesis is true."

No, as pointed out by jostylr this is wrong.

The mathematics of statistical hypothesis testing is almost always defined in terms of a ‘null’ and an‘alternative’ hypothesis, where the ‘null’ necessarily includes the equivalent of ‘no correlations/differences exist’ and the alternative hypothesis does not.

This usually means that the alternative hypothesis is the ‘interesting’ one, which is probably what the researcher would like to occur. Which maybe isn’t entirely intuitive.

But I f you don’t follow those restrictions, the math is meaningless.

If you swap the terms in the math, you’ll confuse anyone who knows the stats, while not really benefitting anyone.

And if you’re not using the math, why refer to a ‘null’ and ‘alternative’ hypothesis at all?

>"the ‘null’ necessarily includes the equivalent of ‘no correlations/differences exist’"

"Null" stands for hypothesis to be nullified in this case, not zero.

Take as an example the standard null hypothesis of the means of two groups being the same. This is a very concrete condition that is amenable to having predictions made from it.

But now let's say that you assume they are different. Well, how different are they? You need to make that choice in order to get predictions. And then your rejection is not a rejection of "being different" but "being different in this one specific way".

>'But now let's say that you assume they are different. Well, how different are they? You need to make that choice in order to get predictions. And then your rejection is not a rejection of "being different" but "being different in this one specific way".'

Yes, precisely. You should predict a specific thing will happen based on some theory and check it with data. This sounds like the science we learned about in school.

Only predicting "there should be some difference" with your theories is more like astrology or something.

if placebo is a sugar pill, maybe it's a sugar sensitivity...

I assume placebo is the same muesli bar used in the other groups (without the supplemented fructan or gluten).

Ah! Excellent point!

if the placebo is a sugar pill then it's an experimental methods failure.

My thought is they aren’t looking at anything other than gluten and fructan. I’m sure there are other compounds common in foods containing gluten that need to be examined and eliminated as causative before publishing a study with a title that implies a conclusion.

Reply on news.ycombinator.com