Hacker News Reader

Comment by indiana-b | Hacker News Reader

I do not understand the problem people have with priors in Bayesian methodology. Yes, it is true that a poor choice of prior can affect results. But classical, frequentist techniques incorporate priors implicitly: a flat prior indicating we have no information other than the data. And just as a poor Bayesian prior based on subjective belief can ruin an analysis, a non informative prior implicitly made can be just as catastrophic. And it is truly a rare case when there is absolutely nothing known about a process, and in these cases, a flat prior is the kind of poor prior that these people are so afraid of.

View on HN

> But classical, frequentist techniques incorporate priors implicitly: a flat prior indicating we have no information other than the data.

That's not completely right. Frequentists don't assume a flat prior, but rather play a minimax strategy that gives a certain worst-case performance across all possible priors. For example, frequentist confidence intervals have coverage guarantees, while Bayesian intervals generally don't. The middle ground is using "objective Bayesian" methods that aim for good frequentist properties.

Right. All human endeavor involves subjectivity in some fashion, Bayesian thinking just lets you quantify it and handle it explicitly.

I believe this is a contradiction. How do you quantify subjectivity?

Let's start with something lighter: how do you categorize subjective states of mind? But we do through words like "happy", "sad", "good-will". Additionally we can make statements as to the degree of sadness, or degree of happiness. Psychologists do this professionally with their survey instruments, but even the layman is capable of saying "I was most happy at my wedding and least happy at my great-grandfather's funeral", which sets bounds with two points of reference.

Oops! Anything with degrees of effect and bounds is basically a finite ordering. And a finite ordering can be given a rank, with numbers. So now we have given numbers to subjective states.

This is obviously quite handwavy, but the core ideas are here. I might be missing something. The big differences are that probability assigns a continuous measure to what we believe is Belief. How do you get an infinite, uncountable number of experiences?

One way is to have the acceptable accuracy be less than an infinite number of digits. That's always going to happen in practice but it's not quite what we're looking for. One other way to do it is to use preferences and expected value. Do you think your knowledge is complete enough such that you'd be indifferent to betting an X amount of money on a roullette wheel with given probability of winning P? Then P is your degree of belief.

In the end there are very good reasons to doubt subjective probability estimates. But as long as one works with bounds and acceptable degrees of error, then you at least get a sense of how it is you're going wrong.

You can maybe gain some sense, but I'm not sure if it can exactly tell you how you're going wrong (except possibly to say, "if you believe these things, it makes no sense to also believe this other thing"; where a thing is a probability value assignment -- the danger is in believing that such analysis can give you positive knowledge [for instance, "I believe these things, so this other things must be the case"). I believe you need a total ordering, not just a finite, partial ordering. Maybe you can say wedding is good, funeral is bad, but how would you incorporate something like the birth of a child, the taste of some food, etc? Basically, you have to reduce all aspects of subjectivity to an integer, which could very well be impossible (making the activity of trying to turn subjectivity into integers highly suspect).

I understand that economic games sometimes use degree of belief, but in the example of a roulette wheel, we can actually count up all of the possibilities and assign numbers based on that. I don't think we can count all of the possible subjective states.

>How do you get an infinite, uncountable number of experiences?

I experience this all the time (though I can't say they are infinite, it is certainly more than I can count).

I lost my reply because the site went down! I'll give a brief one here.

> integers

Or a matrix. Or with some dimension but not others. A total ranking is implausible and lossy, but partial rankings for some traits is tractable: you just have to be careful. If motivated by a decision the use of probability becomes more clear, since you can declare what kinds of errors you can handle and what you can't, relative to the information that you specifically want from an event.

> positive information

This is a problem with statistics, not Bayes. Null hypothesis testing with its p-values and t-tests can only reject a known distribution, not telling you the real distribution without testing for all of them. At that point it can be as prone to GIGO as Bayesian methods are.

There are some statisticians who dissolve the whole Bayes vs Not debate by focusing on the optimisation of loss functions. Although it lacks the philosophical pyrotechnics of subjective probability, in practice it's probably the most reasonable approach: do what works.

You misunderstand what's going on. If I publish a paper people want to know what the data I found suggests and that's it.

A reader can then build there own chain of logic combining several papers with prior knowledge. Further, if someone retracts a paper they can update that chain of logic. But, if a paper is based on another paper that was retracted then it's chain of logic is suspect.

PS: Bayesian reasoning is fine for meta analysis though.

> If I publish a paper people want to know what the data I found suggests and that's it.

What they're going to get, is what your data and your modelling assumptions suggest.

If you're taking just as much care to make the rest of your model unassailably objective, then fair enough. But a prior is usually just one modelling assumption amongst many.

Reply on news.ycombinator.com