Chess - A Quandary for the Game in a High-Tech Era
nytimes.com
nytimes.com
The relevant conclusion: "My model projects that for a 2300 player to achieve the high computer correspondence shown in the nine tested games, the odds against are almost a million-to-one. The control data and bases for comparison, which are wholly factual, show several respects in which the performance is exceptional even for a 2700-player, and virtually unprecedented for an untitled player."
A NYT article from last year on his methodology: http://www.nytimes.com/2012/03/20/science/a-computer-program...
and more details on his site: http://www.cse.buffalo.edu/~regan/chess/fidelity/
Imagine the challenge of excelling in searching the space, evaluating positions, selecting strategy, etc when it'd be legal to creatively combine human and computer capabilities in the game.
There is also Deep Blue[1], which is famous for once defeating Garry Kasparov in a high profile match.
[0] http://www.amazon.com/Blondie24-Playing-Kaufmann-Artificial-... [1] http://en.wikipedia.org/wiki/Deep_Blue_%28chess_computer%29
The comment you're replying to is suggesting teams with two members: Computer A and Human A VS Computer B and Human B.
Thanks for the link to the Blondie24 book!
1) it was with a new unknown method/gadget in his body/clothes
2) it was someone from the audience (they switched off the internet broadcasting so anyone outside could not help him any further)
3) the method he used to cheat was destroyed/hidden before the search (eaten, thrown away, etc)
It's only a little bit harder if you need to communicate in two direction, but - this is chess - you don't need to communicate much data to send out position updates.
Double that if there is two-way communication.
It's a fascinating true story.
A similar device could easily be built for chess computers. Especially since hardware now is much more powerful, and much smaller, and battery technology (while still being lousy) is much better.
What kind of chess computing could you get from, for example, Raspberry Pi? What software would run at reasonable speed on it?
The more obvious approach is to compute the moves on a 'real' computer elsewhere and just transmit the result via some tiny hardware in a shoe or something.
These games were streamed online, so they only needed communication to be one-way (computer to player).
The top computer engines have been strong in closed positions for many years now. If this person did cheat, he most likely was selectively using the engine in this game, or not at all.
There's a reason we don't see computer vs. human matches these days. The computers are too good. You can beat today's elite grandmasters with a free engine running on your laptop.
You may as well suggest Baseball players start playing football because of Bill James and Nate Silver.
So psychology comes into play and that is basically impossible to "solve" unless we have real AI (not anytime soon and the implication would be way broader than "beating Hold'em").
Fixed-limit Hold'em is easier to play because the decision tree is trivial. It's still an imperfect information game but the best bots can already beat some of the best professional players.
But No-Limit Hold'em is another thing entirely: the decision tree is so huge that it's impossible to bruteforce. Impossible as in: unless a major scientific discovery we'll never ever be able to bruteforce it (there's not enough entropy in the know universe to do so).
So we're stuck and waiting for amazing discoveries in the AI field (and psychology field).
Can you be more specific as to what is so impossible about building a smart betting poker bot?
Bot One just plays the odds.
Bot Two just plays the odd, but has a small chance of bluffing - it has a poor hand but plays as if it has a great hand.
I'd be interested to see the results after several thousand hands. And if people could tell the bots apart from people.
This is, obviously, nothing like AI. But sometimes trivial tricks are powerful. Look at the early discussions about ELIZA for an example.
I don't play poker, but my friend tells me that it's fun to play in tournaments because it's not just odds. You get people who have terrible poker faces, who are easy to read. That same friend also plays (and makes a little bit of money) from online poker. I have no idea how the psychology works there.
For instance, "playing the odds" dictates that you bluff. Often. And since this discussion is about HU play, you can change "often" to "constantly."
One way to play is to (attempt to) play game theoretically, or more commonly (and less rigerously defined) "balanced". Your entire actions are defined in such that your opponent, even if they knew your strategy, could not exploit your decisions. Bets have the "perfect mix" (defined by pot size, but also other factors) of value hands to bluffs, so that your opponent is indifferent to calling down/rebluffing. This is what you called "not playing the odds" but, contrary to popular belief, involves bluffing (lots of it!), lots of aggression, and in general may look kinda crazy.
The second way is exploitative poker. This is the more common "not playing the odds" type poker, where you play in a way that maximally extracts value, because you think your opponent is not bluffing enough/betting too big with his range, folding too much, etc. The "too much" is basically "deviating from the correct GTO play". However, to exploit this, you yourself now need to deviate from optimal play, thus opening yourself to exploitation.
There is still a debate between which is "better." Obviously exploitive will, at least in theory, lead to higher winrates, but runs the risk of yourself being exploited. If your "read" is wrong, you could be losing. By playing GTO, you assure you never lose (except your share of the rake), and money naturally flows to you through opponents just basically exploiting themselves.
Actually, it does not, just like it does not come into play for Chess, or for Go, or for Othello.
"Fixed-limit Hold'em is easier to play because the decision tree is trivial"
Trivial? Write a bot for limit Hold'em and you'll see that the decision tree is not trivial. This is not Tic-tac-toe; limit hold'em is a fairly challenging game for computers, and it was only recently that computer players could even compete against the best human players.
"No-Limit Hold'em is another thing entirely: the decision tree is so huge that it's impossible to bruteforce"
Yeah, and the Chess game tree is impractical to brute force, and no Chess playing program actually attempts to do such a thing. Likewise with Go, and likewise with limit Hold'em. So what?
In general, the challenge with Hold'em (limit or no-limit) is not just the size of the game tree, but the branching factor. There are only four betting rounds in Texas Hold'em, and so the game tree is fairly shallow compared to Othello, Chess, or Go. On the other hand, just dealing the community cards adds a large number of branches, and in no-limit games the possible bets a player can make adds even more branching. This limits the utility of approaches based on exploring random paths.
This is not a problem of psychology, it is a problem of algorithms and of game theory.
I am only a casual player of both chess and poker, but I don't see any way that the difficulty in building an extremely good automated Texas Hold'em bot is anywhere on the same order of difficulty as building an extremely good Chess bot.
What am I missing? Isn't the rage with Hold'em lately that the internet players who play mostly straight are cleaning up vs those who try to read people?
Firstly, although it is proven that there is an optimal (game theoretic) strategy to play (Nash), we are not close to 'solving' it. There are attempts to solve limit hold'em, and the current systems beat human players, but they are not close to fully solving the game tree. Attempts to solve big-bet games (NL, PLO) are not even "solved" at a rudimentary level.
Modelling human players is the much easier part. There are winning cash game bots in both FL and NL up to midstakes games, and they are a serious issue. Many use default exploitative strategy that perform well against common opponents.
But the main thrust of my post was that heads up (two players) is mostly solved and you don't seem to address that point. Is that not true? It might not have a closed form solution, but I thought it was approximately solved in that there were algorithms that did no worse than epsilon off ideal play.
I'm not sure if we're talking in the same language here; you cannot lose (beyond your share of the rake) playing a GT strategy, regardless of what your opponent is doing. This is the basis of what GT is.
> But the main thrust of my post was that heads up (two players) is mostly solved and you don't seem to address that point. Is that not true?
No, it is not true. Even for FL hold'em, which has a significantly simpler game tree, the best attempts are still far from a complete solution. Large simplifications/assumptions to the game tree need to be made to make it a manageable size computationally. Much simpler representations for HUFLHE have been solved.
For all other heads up games, researchers are barely scratching the service; I don't even think there's a clear understanding of how to even go about simplifying the game tree to reduce it to a manageable size to even consider solving it.
[I put in the caveat that i (a) haven't read up on the latest in the last couple of years and (b) have not been involved directly with any research projects. I'd love to be corrected from someone who's involved in the latest research.]
What does GT mean here? Not game theoretic, since general Nash equilbria don't have this property. Nash is a strategy where no individual has a motivation to change from the equilibrium.
There are multi-player games that have this property that optimal play always results in statistical winnings, but I doubt that poker is one of them. For example, that would imply that collusion isn't effective against an optimal player.
Heads up is a different matter. There it's fairly elementary game theory that there exists an unexploitable strategy. A brief search says you're probably right, though, about heads up not being solved.
So I maintain everything i said before; HU games are far from solved. Much the same that chess is far from solved.]
The game of poker is symmetrical and zero-sum (ignoring rake).
As you say, "Nash is a strategy where no individual has a motivation to change from the equilibrium." If I were playing a GT optimal strategy, the best you can do yourself is play the same strategy - you are not motivated to deviate. This will be EV neutral to both of us in a symmetric, zero-sum game.
Any deviation you make will either be EV neutral and therefore indifferent, or EV negative. If it is EV negative to you, I gain.
Of course none of this applies to multiway games, we're talking heads up.
> Firstly, although it is proven that there is an optimal (game theoretic) strategy to play (Nash) [...]
The reference to Nash here made me think you were claiming there to be an optimal strategy in the multiway case. I've been talking mostly about multiway ever since. I associate Nash equilibrium with multiway games and wouldn't use the term "Nash" to describe GTO play in a heads up game, even though a Nash equilibrium would be GTO. But maybe this is standard lingo?
Question: So is it the case that there are human players that are measurably better (in a statistically significant way) than the best AI players, heads up?
Some variants of poker are much easier than others. Texas Hold'em is considered to be one of the most strategically challenging poker games and there has been quite a bit of research on Hold'em bots. The current state of the art is this: computer players can defeat human players below the professional level, but professional players still run circles around the computer.
There is a HU FL cash game machine in casinos in Nevada. You can find it in the Aria and other places, i think.
It is rake free (i.e. the house doesn't have an 'advantage'), it is a fair deal. You can go and play it all day 24/7 if you wish. You can play up to $100/200 and $200/400 i believe, and hands are dealt very fast.
Despite some failed attempts, you won't find today any professionals sitting there "running rings" around these machines and making bank.
For a human to learn and become an expert winning player? I doubt it.
There's still the NL frontier, but FL seems to be done.
Computers still overpower humans on 5 0 chess. Even a crappy non-optimized bot I wrote can get 12 levels of the tree plus quiescence essentially instantly.
"Though he was Black, Ivanov achieved a slight edge after 11 moves"
Hint: smart people disable Java applets nowadays ; )