Hacker News Reader

Comment by astroalex | Hacker News Reader

> For instance, say you’re in a race at school. You do surprisingly well and beat most of your classmates. All things being equal, the next time around, you’re actually not likely to do as well, relative to the other runners.

If the events are independent (as I believe the author is assuming at this point), then your performance in the first race has no effect on your performance in the next race, right? Maybe it's just a poor choice of wording, but it comes dangerously close to reinforcing a common misconception about probability (Gambler's fallacy).

View on HN

No, the author is not assuming that. And in fact the opposite is true.

The assumption is that your performance on any given race is some combination of luck and ability. If you perform extremely well, it should be assumed that both were in your favor that time. The next time you'll still have whatever ability is in play, but you are unlikely to have luck.

The result is that your performance is likely to be good, though not stellar. You racing one day or the next are not independent events - your ability creates a correlation. But your performance one day is also not a linear prediction of your expected performance the next.

This is called regression to the mean. It comes into play in everything from stock picking to poker players.

To diagram it out:

  a = ability
  l = luck
  p = performance

    a     l
     \   /
      \ /
       v
       p

Each (ability and luck) has a contributing factor. Your ability is more likely to be consistent (within some stretch of time). Your luck is less consistent. So if you average a 7 minute mile, and you somehow run it in 6, unless your fundamental ability has changed it was luck (weather, mental state, wear on your shoes) that pushed you so far outside your norm, and you should anticipate returning to your average in the future.

What is “luck” supposed to mean in your diagram?

Any contributing factor outside your base ability. Weather (too hot, too cold, just right), mental state (did someone put you on edge, sleep poorly), etc.

So some of those are more or less under your control. Maybe you can control your mental state better than me. Me, a particularly stressful day at work doesn't lend to the quiet mind I need for a run. Those are my 36 minute 5km days. But nicer weather, seeing the geese on the pond as they migrate, can put my mind at ease even on those stressful days and I might pull off one of my faster runs. I can't control that.

so performance is ability plus the contribution of all other factors that are not ability.

tough to argue with that one.

I mean, we can break out luck to be all the various factors. But it can be summed up into one (for the purposes of this discussion). And each of those will still have some mean that they tend toward. The weather, outside San Diego, isn't going to be perfect year round. Your relationships (impacting mental state) won't all be rosy. Your sleep can be interrupted by a neighbor moving in at 3am. Though that last one is atypical. And if it happens, and your race time is 8 minutes instead of the average of 7, you should still expect to be closer to 7 the next race barring similar misfortune.

I think the main point is that performance depends on some factors that we can reasonably expect to remain constant between races ('ability'), and others that are likely to vary quasi-randomly between races ('luck').

Those names are imperfect but the underlying logic is sound: the winner of a race is relatively likely to have both high 'ability' in himself, and higher 'luck' than usual for that particular race. In the next race, his 'ability' will remain high, but he will probably have roughly average luck.

Consider children of immigrants. Look at the ones who are tall, taller than their parents. Are they tall because they got good nutrition, etc.? Many people supposed that they have natural tallness genes and their parents were only shorter because of tough times in the old country.

Now look at their children, the grandchildren of the immigrants. They tend back toward average.

The tall kids were tall because, in part, there is a natural variation and they happened to get on one side of the bell curve. Luck, if you will.

You are committing the gambler’s fallacy. There’s no reason to expect your “luck” (whatever you mean by that) to decrease rather than increase the next time you race.

This is different from saying that you are likely to win again. You’re just as likely to win as you were the first time around, which may be high or low depending on ability.

Drop out the ability part. Let's focus on a pure luck scenario: Take 2 dice (2d6) and roll them. You get a 12. What is more probable on your next roll? Another 12 or a 7?

With regard to the race, if you do really well, much better than your average, without any fundamental change to your ability that's the same as rolling a 12. It's an unlikely (though possible) event that happened to occur. Over a series of races, however, we would not anticipate a repeat of that performance (that is, it's a low probability event, and a series of such low probability events has even lower probability).

The gambler's fallacy goes the other way. It's the belief that if we've had a string of bad luck [0], then we should wager on good luck being around the corner. That is, the gambler assumes a low probability event is "inevitable" after a series of high probability events. Regression to the mean: A high probability event should be anticipated after a low probability event.

[0] In gambling this usually means getting close to the average, that is: the average hand is the lowest valued hand, the rarest hand is the highest valued. So a gambler making this fallacy would wager on seeing a royal flush after several hundred hands of only seeing pairs.

I understand what the GP was saying now (see my sibling comment).

The gambler’s fallacy can apply the other way, e.g. “This die has rolled over 3 many times, so it will probably roll 3 or lower next time.” So that's not really the issue.

Your dice example is not illustrating the point the GP is trying to make. It’s just comparing the probabilities of two outcomes of the same distribution (the sum-of-two-dice distribution).

This has nothing to do with the selection effect the GP is talking about. That would be recording the value of the die that rolled highest (i.e. taking the maximum over dice) and then focusing on the same die next time (i.e. not taking the maximum over dice).

It’s comparing the probability distribution of max {X_1, X_2, X_3, ...} with the probability distribution of any individual X_i. If A is sampled from the former and B is sampled from the latter, P(B < A) > P(B > A).

No, I am not committing the gambler's fallacy. Though I can see why you might think that. There are enough things that sound similar.

I'm instead talking about a more subtle detail. Which is that selecting a group based on performance results in selecting people in part for having been lucky. They were lucky to have done that well that time, were lucky to be in your group, and their future performance probably won't be as good.

The result is called regression to the mean. See https://academic.oup.com/ije/article/34/1/215/638499 for an example of statisticians talking about it. See http://onlinestatbook.com/stat_sim/reg_to_mean/index.html for a more introductory tutorial. And see http://wmbriggs.com/post/63/ for an example of an article discussing this counterintuitive phenomena in the context of sports. (Namely the "Sports Illustrated curse" - the future performance of athletes whose performance was good enough to get them on the cover of Sports Illustrated drops after their article appears.)

I think I see what you mean now.

If you roll 20 dice and select the highest, the next time you throw that very same die it’ll probably have a lower value, simply as a consequence of outcome-dependent selection + a fixed probability. It’s not that the probability has changed, just that you didn’t choose the first outcome according to the true generating distribution.

Comparing this highest value with the highest value across all coins when you toss them all again a second time is a different matter. It’s no more likely to be lower than higher.

It's best to avoid using the term “luck” altogether in discussions of probability, since it’s easy to misinterpret.

That's exactly the right idea.

If you have a competition where everyone flips coins, and you are eliminated as soon as you flip a tail, and you win, it's likely that you will also win the next time. This is not because the events are not independent, but because you are always unlikely to win.

Likewise, if you get a royal flush in a hand of poker, you probably won't get as good a hand next time. Again, this is not because the deals aren't independent events, but because it is always unlikely to get a royal flush.

Good points.

> Likewise, if you get a royal flush in a hand of poker, you probably won't get as good a hand next time.

I understand what you mean, but I think that most people who haven't taken a probability class would misinterpret this. A sentence of the form "if [this] then [that]" is usually meant to express dependence. Don't you think that most people would interpret your sentence as: "Because I got a royal flush this time, I probably won't get one next time."?

I think the average pop-sci article reader will get confused by this.

it's really hard to map subtle statements about probability to natural English sentences. pretty much any non-awkward formulation of that sentence is going to admit misinterpretations.

> it's likely that you will also win the next time. This is... because you are always unlikely to win.

Did you mean to write “it’s not likely that you will also win next time”?

I did. Unfortunately the editing window is closed. Thank you.

I don't think it's phrased to correlate the second outcome dependent on the first. That part of the article was referring to the racer regressing to their mean. With the assumption that they overperform in their first race based on the phrasing "surprisingly well", then the conclusion is still correct because the likelihood of them performing to their average performance level (whatever that may be) is still more likely than overperforming.

I was surprised at this passage, too. It begins by quoting the Oxford dictionary, which always raises alarm bells that the author's ideas on the topic are inchoate, but the wording of the 'racing' example is an unforgivable blunder. The 'mean' referred to in 'regression to the mean' is the mean of the individual, not of the pack.

I don’t see why you think that. Let’s say each of your classmates has a number of dice representing their innate ability, and rolls all of them. You have 5 dice, do surprisingly wel and roll four sixes and a four for a total of 28, beating some classmates who have 6, 7, or even 8 dice.

If you repeat this, it’s unlikely you will do similarly well.

Reply on news.ycombinator.com