Hacker News Reader

Comment by civilized | Hacker News Reader

Nonsense. Frequentist statistics is (relatively) easy to understand. If people can't be trusted with it, they certainly can't be trusted with Bayesian statistics.

This is more of an argument not to teach anyone statistics at all, other than some unspecified elite that the author feels qualified to learn it.

By the way, the author has some interesting obsessions:

https://twitter.com/FamedCelebrity/status/155513494784911360...

https://twitter.com/FamedCelebrity/status/155516247586318336...

https://twitter.com/JohnZmirak/status/1554969385860796420?s=... (retweet)

View on HN

Let me introduce you to Gerd Gigerenzer's paper "The Null Ritual", in which he brilliantly shows that almost nobody understands the rituals of frequentist statistics [1].

"How many students and teachers noticed that all of the statements were wrong? As Figure 1 shows, none of the students did. Every student endorsed one or more of the illusions about the meaning of a p-value. One might think that these students lack the right genes for statistical thinking and are stubbornly resistant to education. A glance at the performance of their teachers, however, indicates that wishful thinking might not be entirely their fault. Ninety percent of the professors and lecturers also had illusions, a proportion almost as high as among their students. Most surprisingly, 80% of the statistics teachers shared illusions with their students."

[1] http://library.mpib-berlin.mpg.de/ft/gg/GG_Null_2004.pdf

I don't disagree that many people struggle to properly interpret frequentist statistics. But adopting a Bayesian perspective isn't a magic talisman. The number of people who are competent Bayesians but don't understand frequentist statistics is zero, and no pedagogical changes will improve that.

I don't agree on this one.

The problem with frequentism and the null ritual is that it makes statistics easier (just do this one test, read this one number) and renders some kinds of mistakes somewhat harder (publishing a single false positive).

At the same time, it makes some bad mistakes much easier, most notably ignoring power, false positives, the garden of forking paths, type S errors and publication bias.

The inherent problem is that the null model is not what people assume it is, and the method (or at least the established canon of approaches) don't make you think about it.

If you use bayesian methods, you're pretty much forced to spend more time considering the effect size and credibility of your results, and you're basically required to report them.

This means that even non-competent bayesians probably have a better contribution to cumulative science.

It sounds like you read Andrew Gelman's blog. You might be interested to review this post: https://statmodeling.stat.columbia.edu/2012/11/10/16808/

Frequentist statistics does not force you to accept any hypothesis test with a result p < 0.05 as definitive proof of something. It does not forbid considering prior probability of a result. It just doesn't formalize the consideration of prior probabilities because it is hard to distill this consideration into a formal recipe.

Everyone agrees that Bayes' Rule is valid and important, the question is when and how best to use it.

> If you use bayesian methods, you're pretty much forced to spend more time considering the effect size and credibility of your results, and you're basically required to report them.

You're not forced to do those things well. Any scientific method can be cargo culted.

Yes I think Gelman is doing great work, as is e.g. McElreath [1].

The problem that I care about is not whether frequentist statistics can be taught and used well. They can, and I try in my teaching to do so.

The problem is that empirically, frequentist statistics is a fig leaf for a ton of extremely problematic work. And pushing bayesian thinking is currently our best chance to fix this, because it's easier to do a shift in the mental framework than to fix the perception of an existing framework.

[1] https://xcelab.net/rm/statistical-rethinking/

I agree that more Bayesian thinking is needed. But I suspect that pushing the technicalities of Bayesian analysis (MCMC etc) would perversely lead to even more cargo-culting, as people would struggle with the technicalities and look desperately for quick fixes.

Bayesian methodology benefits from having relatively much more statistically sophisticated practitioners, which leads to an optimism bias when we imagine how it would scale up.

Perhaps you don't see it this way, but I take our eventual convergence as a sign that we don't actually disagree. It's rather that the problem we're starting from is different. Thanks for taking your time for this debate!

Likewise! This is a tricky problem and requires a lot of patient thought.

I guess it depends on the context in which it is applied. It is anecdotal, but in the industries where I worked, people trying to introduce Bayesian statistics did not have a higher chance of their results being interpreted properly. If you do A/B testing and don't do pre-registration or power analysis, what are the chances that you can/will be able to explain the nuances of probabilistic reasoning ?

I agree w/ the parent poster than the fundamental issue is probability: if you are talking w/ people w/o background in stats, you will have a really hard time to go beyond a true/false statement.

Moreover, one of the most effective (in $ terms) application of statistics in recent times is A/B testing. While you can do "Bayesian A/B testing", the basic methodology is fundamentally frequentist. Mistakes there can be hedged through better tooling / UX (to avoid peeking, etc.), as effectively as using Bayesian statistics.

Hum... You are basically equating frequentism with the null ritual. The article itself is basically equating frequentism with the null ritual.

Ok, that explains why people bother spend so much energy badmouthing frequentism, but it's a very bad framing anyway, bothering a lie. Frequentism is not the null ritual. In fact, it's almost completely compatible with bayesianism, the one large difference being the freedom to set priors before doing your analysis.

If the article was titled "It's time to stop teaching the null ritual to scientists", nobody would even disagree.

> Frequentism is not the null ritual. In fact, it's almost completely compatible with bayesianism, the one large difference being the freedom to set priors before doing your analysis.

This may be true of "frequentism" in a strict sense, but many statistical methods in common use that are often described as such (including, arguably, NHST) are not consistent with reasonable versions of the likelihood principle https://en.wikipedia.org/wiki/Likelihood_principle . In a sense, one might feasibly argue that these methods are not even properly frequentist.

No one said it’s a magic talisman. But Bayesian probability is far more in line with most peoples intuitive understanding about probability and statistics than frequentist probability.

There will always be no instances where individuals err in the way they apply statistical methods. But that doesn’t mean that there is no value in moving the “default” in statistics to a more intuitive methodology from one that is so obtuse that it’s common for relatively advanced practitioners to stumble over it

The problem is deeper than people just struggling with frequentist statistics. It's difficult IMHO to actually reason properly about frequentist results without also having a grounding in Bayesian statistics. For example, in the Gigerenzer et al. paper cited above, to properly understand why statement #6 is false, a person has to understand that they've confused p(D|H0) with 1 – p(D). That's why it's not surprising IMHO that cookbook approaches to frequentist statistics without a nontrivial Bayesian component, as are often found in social science-focused stats courses, inevitably end up with limited or skewed comprehension.

I vehemently agree with this. It's the OP's "stop learning frequentist statistics" that I vehemently disagree with.

The difference between Bayesian and frequentist statistics is that they answer different questions. People are often more interested in the questions that Bayesian statistics answer, and so they often misinterpret frequentist results to also answer those questions.

It's possible that it can happen the other way around as well, but my impression is that it happens less often.

I don't see the relevance of his personal beliefs or prejudices.

Well, the "article" is mostly an opinion piece, it could as well be a Twitter thread. So, at least, there is some value when evaluating a personal opinion in the context of other opinions expressed publicly by the author. I'm not saying this is the case, but I don't think is invalid either.

Of course it’s invalid. Either there is something about the substance of the article that is debatable or there isn’t. Other opinions are completely irrelevant.

If I write an article about the benefits of X drug and have expressed publicly by other channels my support for the company that produces such drug, I think is relevant for the discussion of the article I just wrote. That's why I said that may be it is not the case here, as I my self don't see any relationship between the linked tweets and the article. But I don't think is to crazy to think that two related opinions can be analyzed together regardless of the channels of distribution.

If you write an article about the benefits of X drug, you have presumably made arguments that can or can not be supported with available evidence. Your other interactions with the company producing the drug are irrelevant.

That's not very Bayesian of you.

You need to review your understanding of Bayesian statistics then

Browsing his personal website reveals frequent mistakes in science, statistics, and probability theory. I don’t think I’ll spend any time on this paper.

It is not just his beliefs it is also how he seems to get fooled by Russian tabloid style fake news.

> Frequentist statistics is (relatively) easy to understand.

Easy to misunderstand. NHST creates false beliefs, and that's even when you do everything correctly. If you accept NHST, you believe in "psi", because Bem has proven it.

The twitter links you cite actually make me respect the author more, it shows for example he's a contrarian in a situation where that would cost him a lot, an attitude that I highly respect.

Alas, the article is badly written and unprofessionally laid-out, and, ironically given the title, isn't written with a non-statistician in mind.

There's a whole cottage industry of audience-captured contrarians who rely on "contrarian trust" to sell you the opposite of whatever the Establishment is selling.

It's not any more thoughtful, it's just the opposite.

There's a not-so-subtle failure mode where you go from "Some of those against the Establishment are cranks" to "All of those who go against the Establishment are cranks", it feels more clever and satisfying than "I really really like the Establishment and can't tolerate it or its opinons being wrong", but it's indistinguishable in input-output behaviour.

If you are talking about his statistics opinions, I don't understand enough statistics to know if it's right or wrong or not even wrong. But his other claims gives him a substantial respect-boost regardless of whether he's right or wrong. I simply like contrarians more.

This is all uninteresting to me because contrarian versus Establishment is a pretty poor indicator for truth. Leaning on that is like the famous drunkard leaning on the lamppost, more for support than illumination.

The yellow flag in those tweets is the hyperventilating culture warrior attitude. It speaks poorly of ability to humbly and dispassionately analyze topics, regardless of which side it comes from.

>contrarian versus Establishment is a pretty poor indicator for truth

False. It's a pretty reliable proxy if the Establishment is known or plausibly suspected to benefit from falsehood and\or misrepresent truth. If you have a reliable predictor of falsehood, then something which always opposes it is a reliable predictor of truth. I don't know if the statistics Establishment is one such example (probably not), but the trans-activism and war-media Establishments certainly Are reliable predictors of falsehood, and have been shown to be on multiple occasions.

That's why I like the author's opinions on those topics more than on statistics, even if I'm not necessarily certain of their truth. He's going in the rough general direction of truth.

>hyperventilating culture warrior attitude. It speaks poorly of ability to humbly and dispassionately analyze topics, regardless of which side it comes from.

False Again. Plenty of smart people have this attitude, the notorious example off the top of my head is Nassim Nicholas Taleb, incredibly arrogant and quick to ungraciously fail into combat under criticism - yes - but do you actually want to take him one-on-one in statistics? For a more distant example, see Newton's hilarious pettiness in dealing with rivals. Your heuristic yellow flag would have lead you to dismiss him faster than you can say "Universal Gravitation", and we both know you would be utterly and irredeemably wrong.

> By the way, the author has some interesting obsessions:

> https://twitter.com/FamedCelebrity/status/155513494784911360...

> https://twitter.com/FamedCelebrity/status/155516247586318336...

> https://twitter.com/JohnZmirak/status/1554969385860796420?s=... (retweet)

None of those twitter threads have anything to do with statistics pedagogy at all. What sort of irrelevant, ad hominem argument are trying to make here?

Reply on news.ycombinator.com