Hacker News Reader

Comment by usgroup | Hacker News Reader

From the question:

“However, it is very important that the uncertainty in the number of trials is taken into account because over-estimating a fraction is a costly mistake.“

Seems fairly clear to me that you’re supposed to use a lower bound estimate to take into account variance on the fraction due to the number of trials in a way to bounds the chance of over estimation.

Further, there is no need for a heuristic when there a several statistical models for this exact problem with clear properties. Some are given in the answer.

View on HN

Out of context, the expression "the uncertainty in the number of trials" would refer to missing knowledge in terms of how many trials actually ran.

In the context of the post this doesn't make sense, so the reader is left to hypothesize what the writer actually meant.

I agree it could be clearer but as a general rule, if you find an interpretation under which the question doesn’t make sense, try considering another interpretation.

I think "uncertainty due to the number of trials" would be clearer.

Uncertainty in the number of successes due to the number of trials, would be even better.

That would also be an incorrect phrasing. This entire thread is a good illustration of the difficulty of speaking precisely about probabilistic concepts.

(The number of successes has zero uncertainty. If you flip a coin 10 times and get 5 heads, there is no uncertainty on the number of heads. In general, for any statistical model the uncertainty is only with respect to an underlying model parameter - in this example, while your number of successes is perfectly known, it can be used to infer a probability of success, p, of 0.5, and there is uncertainty associated with that inferred probability.)

I guess I struggle to understand why under estimation is worse than over estimation, when the final result is a ranking. It seems like they're equally likely to produce an incorrect ranking!

Reply on news.ycombinator.com