Hacker News Reader

Comment by travjones | Hacker News Reader

I agree with #2, but I don't agree with #1. Why would one need a p-value to make a decision about the significance of a change in a dependent variable? Confidence intervals provide more information than p-values, and you can use them to identify significant effects. Moreover, p-values do not indicate the effect size that was observed. Thus, a very small effect in a very large sample can have a low/significant p-value.

View on HN

p-value testing and confidence interval testing are not interchangeable?

also, p-values are used to determine the number of effects in a mixed model, or the number of factors in an ANOVA, etc. Sometimes confidence intervals on abstract parameters can be a bit less useful.

You are correct, they are not interchangeable. I didn't suggest that they were. I was offering CIs as an alternative for examining effects in general (e.g., comparing two groups); however, your choice of data representation will always depend on the question that you're asking. Nonetheless, p-values in and of themselves provide no information about the magnitude of an effect. Why not show the actual data?

I guess you mean clinical significance.

http://www.bmj.com/content/348/bmj.g2130

This is a secondary issue. For most non-stats people, I wouldn't recommend the confidence interval as a measure of clinical significance. I'd rather look at the actual distribution of the two effects and view quantiles of the predictive interval.

And, yes, in hypthoesis testing, p-values and confidence intervals are generally interchangeable. All they speak of is the overlap of the sample distribution of means, really.

Reply on news.ycombinator.com