Hacker News Reader

Comment by oneoff786 | Hacker News Reader

The fancy deep learning poker bots that beat pros don’t even consider past opponent behavior.

If the bot is playing a Nash equilibrium then it doesn't need to devise the opponent's strategy. It's guaranteed to tie the opponent in the worst case (the opponent is also playing Nash equilibrium) or else win.

https://en.wikipedia.org/wiki/Nash_equilibrium

However, if the opponent is playing sub-optimally you can win _more_ by deviating from Nash equilibrium to take advantage of their specific strategic shortcomings. The risk in doing so is if you are wrong estimating their shortcomings.. you are no longer guaranteed a tie in the worst case scenario.

You can see this play out with a small toy game and a card calculator. Suppose a two player game of Texas Holdem where each player has 8x big blinds. The first player must choose to either go all in or fold. The second player must choose to either call the all in or fold. Players must pick a range of hands to perform either action prior to looking at their cards.

https://openpokertools.com/range_equity.html

Suppose a silly strategy by the second player of folding any hand except pocket aces. You'd be wise as the first player to go all in with any hand and pick up a free big blind with 99.5% certainty. However going all in with every hand is also an easy strategy to take advantage of...

(You can actually go back and forth maximally exploiting the other player's strategy and you will eventually reach Nash equilibrium)

On its face that seems impossible to be true. Given any bet you could calculate the probability that the bot would fold given an all in raise if you had perfect knowledge of the bot and abuse it

At some point I want to write a long-form post with code proving my statements above. I walked through this scenario a long time ago using https://www.flopzilla.com/holdeq.html to find the equilibrium but it'd be neat to write open source code that does the number crunching and that way others could actually audit the proof.

The short of it though is there exists a perfect range of cards the bot can play where it neither folds enough for you to bluff to gain an edge, nor does it call enough that you can wait for better cards while bleeding chips to gain an edge.

The reason I think this isn’t theoretically possible is that the bets aren’t binary. You have to specify a number. So with knowledge of the bot you could probably set up some strong priors on what they have.

I say this despite also claiming the top DL bot does this anyway and does very well.

If you can abuse knowledge of the bot's strategy then the bot is by definition not playing a game theory optimal strategy.

If you're talking about Libratus (the CMU AI that beat poker pros in heads up no limit), then that's not true I believe.

During the competition, Libratus used 4 million core hours on the Bridges supercomputer for analyzing it's prior gameplay and changing strategy.

> During the tournament, Libratus was competing against the players during the days. Overnight it was perfecting its strategy on its own by analysing the prior gameplay and results of the day, particularly its losses. Therefore, it was able to continuously straighten out the imperfections that the human team had discovered in their extensive analysis, resulting in a permanent arms race between the humans and Libratus.

From https://en.wikipedia.org/wiki/Libratus

Definitely not true for the top bots.

Reply on news.ycombinator.com