Comment by hnuser123456 | Hacker News Reader

Comment by hnuser123456 | Hacker News Reader

Okay, apparently this is the core of the debate?:

Frequentists view probability as a long-run frequency, while Bayesians view it as a degree of belief.

Frequentists treat parameters as fixed, while Bayesians treat them as random variables.

Frequentists don't use prior information, while Bayesians do.

Frequentists make inferences about parameters, while Bayesians make inferences about hypotheses.

---

If we state the full nature of our experiment, what we controlled and what we didn't... how can it be a "degree of belief"? Sure, it's impossible to be 100% objective, but it is easy to add enough background info to your paper so people can understand the context of your experiment and why you got your results. "we found that at our college in this year, when you ask random students on the street this question, 40% say this, 30% say this..." and then considering how the college campus sample might not fully represent a desired larger sample population... what is different? you can confidently say something about the students you sampled, less so about the town as a whole, less so about the state as a whole...

I don't know, I finished my science degree after 10 years and apparently have an even mix of these philosophies.

Would love to learn more if someone's inclined.

You can never state the full nature of your experiment. Even the simplest experiment under the most controlled conditions has a bunch of unknown unknowns - you can never be certain that you didn't get a bad batch of reagents or a bit flip in your data or just royally screwed something up. Unless you're omniscient, there are always unknown unknowns lurking somewhere in your method.

In the case of your survey, you can't really state anything about the students you sampled with absolute confidence. You might have been asking a question about something that a lot of people are inclined to lie about. You might have been subconsciously biased and subtly influenced people towards the "right" answer. Slight inconsistencies in the wording of the question might radically alter how people respond. The student you tasked with conducting the interviews might have just stayed in bed and fabricated the data.

Bayesianism gives us an incredibly powerful framework for reasoning about this innate and unavoidable fog of uncertainty; frequentism largely pretends that it doesn't exist. The ongoing replication crisis shows why this is not merely pedantry, but the single most urgent issue in science.

the replication crisis is more about poor experiment design, low-n studies, and cherry picking results.

and that's not to say cherry picking is bad. lets say you set up an n=5 experiment and you drop your instrument on the floor on experiment #4. please cherry pick that one away.

>frequentism largely pretends that it doesn't exist. The ongoing replication crisis shows why this is not merely pedantry, but the single most urgent issue in science.

If you mean that Frequentist methods have no way of dealing with parameter uncertainty then your statement is false.

If you mean that some people who use Frequentist methods don't deal with parameter uncertainty then it may sometimes be the case.

Well even for simple things there's a large difference. Say you toss a coin N times and observe heads x times. What is the probability of your next toss coming up heads?

A frequentist arguably would say the question doesn't really have any meaning since probabilities are about long run frequencies of things occuring. They might do various tests or tell you the probability of that outcome under various probabilities for heads.

A Bayesian would make an initial assumption about the probability of any given probability, and then compute a posterior using the likelihood function the frequentist may have, and give you a distribution for what you should believe about the what the true probability of heads is on your next coin toss.

In general, the latter is more meaningful and informative. There's also pretty good arguments that any coherent method of representing credences is isomorphic to probability, see Cox's theorem.

x ~ Binomial(N,p) and you wish to estimate p.

Here are a whole collection methods for how to estimate p and calculate a confidence interval for it: https://en.wikipedia.org/wiki/Binomial_distribution#Confiden...

One of the methods is Bayesian; the rest are not.

Not mentioned in the list, but you can also use likelihood ratio intervals calculated from a likelihood profile: another Frequentist method.

None of the methods -- including the Bayesian, requires an informative prior.

> If we state the full nature of our experiment, what we controlled and what we didn't... how can it be a "degree of belief"?

Here's a question for you: what's the millionth digit of Pi? (In the standard decimal expansion, of course.)

This is a question (kind of an "experiment") which literally has only one, constant answer, that is theoretically knowable. But unless you search online, or happen to know the answer, you actually don't know which of the digits 0-9 it is. And I can just as easily ask about the 10^100th digit of Pi, which is, again, a constant - and yet no one knows what it is.

So using the Frequentist approach to statistics doesn't make much sense - there's no repeated experiment with possible different outcomes.

But there is a real sense in which your answer should be "it's one of the digits 0-9 with 1/10 probability each". That answer makes sense, because the answer isn't unknowable, just unknown, and probability reflects your lack of knowledge and degree of belief.

The repeatable experiment is asking for different digits of different normal irrational numbers. If repeated enough times, the answer will be each digit 0-9, each one about 1/10 of the time.

I think Bayesian methods have made ground in sciences such as Sociology, Psychology and Ecology, which are mostly observational, but still attempt to make models with intepretable parameters.

With observational studies, representing confounders and uncertainty is a primary concern, because they are the most important source of defeater. Here, Bayesian software such as brms, Stan, pyMC, become a flexible way to integrate may sources of uncertainty. Although, I suspect methods like SEM still dominate for their use cases.

Personally, I find myself using Bayesian methods in a similar bag-of-tricks way that I use Frequentist methods mostly because its difficult to believe that complex phenomena is well described by either, so I use whatever makes the case best.

Reply on news.ycombinator.com