Gambler’s Ruin Problem [pdf] | Hacker News Reader

Gambler’s Ruin Problem [pdf]

columbia.edu

61 pointslawrenceyan27 comments

I studied Gambler's Ruin while taking probability as an undergrad (and studying to land a quanty job after college). For folks who enjoy these types of puzzles, another similar exercise that I spent an entire afternoon trying to solve was the following:

You have 52 playing cards (26 red, 26 black). You draw cards one by one. A red card pays you a dollar. A black one fines you a dollar. You can stop any time you want. Cards are not returned to the deck after being drawn. What is the optimal stopping rule in terms of maximizing expected payoff?

(source: http://puzzles.nigelcoldwell.co.uk/fourteen.htm)

From some quick "just several minutes" beer-laden kit bashing in my brain:

1. Draw 26 cards.

2a. Equal to or greater than $26? Stop.

2b. Less than $26? Draw 13 more cards.

3a. Equal to or greater than $1? Stop.

3b. Equal to or less than $0? Keep drawing until equal to $0, then stop.

I'm assuming I want to always stop at a non-negative value, else I'd pay the house on the way out. If ending with a negative value has no consequences beyond the game itself, then just stop at 2b after drawing.

Not sure I followed what you meant by >= $26 in 2a. The max profit one can make is $26, by drawing 26 red cards in a row, never more.

You are correct though that you'd never want to stop on a negative number, as you always can execute 3b

For the same reason, you’d never intentionally stop at $0 (unless the deck is exhausted). You can always get back to $0 by drawing the rest of the deck, and there’s a 50% chance of going up a dollar on the next card.

If your sum is currently positive, then the chance of going up a dollar is below 50%.

Looking back, I think I got my numbers mixed up (thanks, beer!). $12 or $13 is probably the number I'm looking for, not $26.

Anyway, my logic is if I am at or higher than the median after I have drawn half the deck, then that means there are more black cards than reds in the remaining deck and I have a greater chance each draw of losing a buck than gaining.

So it is in my best interest to pull out at that point, than to try my luck against ever worsening odds.

The EV to drawing another card is going to be tied to the probabilities of continued random walks eventually landing on higher scores than you currently have. You stop as soon as the EV is negative (or 0 if you’re not a gambler).

That calculation is presumably the tricky part… integrating over Pascal’s Triangle or something. But it must be positive to start: If the first card is red we have already gained, and if it’s black we’re at worst back where we started.

This reminds me how I was introduced to this topic, the St Petersburg paradox. It was after a discrete 2 final. The professor heard us talking about a competition from our Quant finance club and decided to ask us this problem. Essentially asking the same thing but with a dice, so with replacement, with bigger values and asking what logic our stop algorithm would be to maximize our return. And what would happen if the gain and loss weren't equal with something like +4,+3,+2,-1,-2,-3 as well aas with -4,-3,-2,+1,+2,+3. We ended up staying an extra 2 hours discussing it.

Very nice puzzle! A quick Python script that solves it as it's difficult to do with a pen and paper:

  #awesome feature from Python 3.9!
  from functools import cache
  
  @cache
  def ev(score, cards_left):
      if cards_left == 0: return 0
      up_cards_no = (cards_left - score) / 2
      p_up = up_cards_no / cards_left
      p_down = 1 - p_up
      return max(score, p_up * ev(score + 1, cards_left - 1) +  p_down * ev(score - 1, cards_left - 1))

  >>> ev(0,52)
  2.6244755489939253

Of course we should continue if the current score is lower than ev. Here is the script to find exact thresholds at which we should stop drawing more cards:

  def find_thresholds():
      print(f"EV of tha game: {ev(0,52)}") # fill the cache up
      for i in range (0,10):
          thres = 52 - i
          while ev(i, thres) > i:
              thres -= 2
          print(f"At score {i} you should stop drawing with {thres} cards left")

  >>> find_thresholds()
  EV of tha game: 2.6244755489939253
  At score 0 you should stop drawing with 0 cards left
  At score 1 you should stop drawing with 3 cards left
  At score 2 you should stop drawing with 8 cards left
  At score 3 you should stop drawing with 17 cards left
  At score 4 you should stop drawing with 28 cards left
  At score 5 you should stop drawing with 41 cards left
  At score 6 you should stop drawing with 46 cards left
  ...

Being up 6 or more we should always stop. The last interesting number is being up 5.

I am not math enough to begin to understand this paper but nonetheless I was instantly reminded of the Kelly criterion, which is super nifty: https://en.wikipedia.org/wiki/Kelly_criterion

The main difference is that Kelly has a multiplicative assumption and the paper has an additive assumption. The Kelly criterion is often computed as a proportion of the initial bankroll, whereas this paper handles non-zero chances of losing a _fixed_, positive amount of money (i.e., a varying proportion of your bankroll, and that proportion increases precisely when you would least want it to -- when you're doing poorly).

Classic recursive or iterative problem that an undergraduate Statistics major needs to be able to derive directly not using formulas. Its wild how the chance of going broke is your money in bet units above both sides money at 50/50 and the odds go dramatically worse with any house edge.

Is it because one can get ruined by hitting 0 but a "win" is subjective because there's no absolute upper bound? If one could go in to negative at 50/50 odds, then the game will dance around that $1 starting point forever?

If there's no absorbing states and it's the cumulative sum of infinite coin flip (-1, 1), it'll visit every integer an infinite number of times with probability 1 a.s..

So in real-world, the gambler would owe all the money in the world at some point. But also win all the money in the world at another point! And then cash out and he wins? Sounds beatable.

The gambler will owe all the money (and then some) with probability 1 a.s., but it's not guaranteed to happen because a.s. implies it's not a certainty.

> Its wild how the chance of going broke is your money in bet units above both sides money at 50/50

It's actually quite simple. In a fair game you expect to leave with as much money as you started with. initial money = expected final money = p * all money

To be clear, the reason why expected final money is `p * all money` is because the only "absorbing states" are the win state and lose all money state, so it's `p * max money + (1 - p) * 0`, right?

Also, it's not quite clear to me why in this case initial money = expected final money. For the related game where you go double-or-nothing on each bet, it's evident this holds since the expected value of your next turn is the same as your current (I believe it's called martingale property), but that doesn't hold for the game in the article where you win +1 or -1 unit each bet, since if you e.g. have 4 units and the win state is 5 units, you're more likely to win than lose.

Edit: nevermind, I think it is a martingale since if you're at 4 units the expected value of the next state is 0.5*5 + 0.5*3 = 4. I guess my confusion stemmed from the fact that even though the long-term expected value is unchanged, you can still have good odds of winning (which is balanced by a small chance of a catastrophic loss), e.g. 4 = 0 * 1/5 + 5 * 4/5.

I recently came across the concept of Gambler’s Ruin in Taleb’s “Skin in the Game”. It’s… definitely written in Taleb’s style, but I’ve noticed myself relating all sorts of things back to the book recent. Worthwhile, overall.

https://en.m.wikipedia.org/wiki/Skin_in_the_Game_(book)

Reminds me of the St. Petersburg paradox [0], which was one of the first problems I simulated in Python to learn the language. Pretty cool exercise.

[0]: https://en.wikipedia.org/wiki/St._Petersburg_paradox

Studying CS a while ago, we had a very practical statistics and now I'm struggling to understand more theoretical proof-based papers like this one. Is there a good statistics/probability book out there to brush up on my statistics and understand papers like this better?

Here is another solution. Assume a head multiplies your money by `r = q/p', and a tail divides by `r'. This bet is fair (the expected value of a dollar is a dollar: `p r + q 1/r = q + p = 1').

Start with a dollar, after several tosses you have `r^(head-tail difference)' dollars. The game ends if head-tail difference is `N - i' (with probability `x') or if head-tail difference is `-i' (probability is `1 - x'). The expected value is still a dollar:

1 = x r^(N - i) + (1 - x) (1/r)^i

Solve this equation for x.

Studying for quant interviews lol

the URL sends me to a object not found page

This is relevant for trading and the reason why people adopt the 1% risk rule.

What's the 1% risk rule?

Never risk more than 1% of your capital. At any one time your position(s) size can be 100% of your capital but the most you can loose from the trade(s) is 1% of your capital.

So after winning a trade(s) you increase the amount you are risking and after loosing you decrease the amount you are risking.

Reply on news.ycombinator.com