Hacker News Reader

> encode the difference between what is predicted and the actual data

Minor nitpick: In the idealized model there is no single prediction that you can take the difference with. There is just a probability distribution and you encode the actual data using this distribution.

Taking the difference between the most likely prediction and the actual data is just a very common implementation strategy.

View on HN

How is "the most likely prediction" not a "single prediction that you can take the difference with"?

What if there are two (nearly) equally likely predictions? What if there are N, and they are not close together?

"A single prediction that you can take the difference with" makes some assumptions about the shape of your distribution, at least under any reasonable model of "code the difference" where, e.g., probability decreases as the difference gets larger. These are often very good assumptions, to be fair.

Reply on news.ycombinator.com