Hacker News Reader

Comment by jhales | Hacker News Reader

I just saw Norvig give this talk. The quote cuts off before he offers his explanation: the 1.5 means 'trust this data set more.' For whatever reason (e.g. bigger sample size) the probabilities from that set tend to be more accurate.

It's not some sort of deep mystical counter example. It's a clever tweak that comes from the observation that empirical observations are not uniformly reliable.

View on HN

Given accurate probabilities as input, Bayes always gives the best result. That's a theorem. So the fact that this tweak improves the performance probably means that either the english model probabilities are systematically too low, or the translation probabilities are systematically too high. My guess is the latter.

As it's probability^1.5, it makes the probability lower, and not higher, i.e., x^1.5 < x if 0 <= x <=1.

I think this might be a tweak due to data scarcity. To compute p(e), there's tons of data to build a model. On the other hand, p(e|f) requires for you to get a lot of parallel corpus (texts that's written both in English and Foreign language) which is not easily doable.

As a result, p(e)^1.5 * p(e|f), lowers p(e) as it's supposed to be too high compared to p(e|f), imho.

I think it's more helpful to think of it as a a gamma correction with γ > 1, rather than saying higher or lower. Or as I said in another comment lower temperature, in that it concentrates probability around its peaks.

Good point, my mistake :)

Reply on news.ycombinator.com