Robot wars break out on poker sites | Hacker News Reader

Robot wars break out on poker sites

theregister.co.uk

83 pointsmootothemax75 comments

This cat and mouse game is very interesting. I wrote a bot years ago that was easy to implement, since you could hook into the poker clients and listen for the window handler events. The more recent versions actually implement the console as graphics and the buttons as custom controls so that you can no longer do this. The modern bots use OCR to read the table play, so the sites responded again by changing it up.

Having a single bot in a tournament is not bad per se, since they are easy to identify (they do not respond to player rage, they shove or call on hands that may be +EV isolated, but in tournament play are not, and they are the player who come final table sit all quiet and do not respond to deals) - the prob arises when bot networks join a single tournament and collude - especially on the double or nothing tournaments (where if you finish in the top half you get double your money, or nothing)

Stars and FTP are very agressive with combating the problem. I receive refunds all the time because of previous games that have been investigated, so it is great that other players can rely on their work (and it is good to login and find that you have extra money because they are doing their job). These sites are making a lot of money on rake but they are definitely re-investing it to protect the players. They have hired some of the best security guys in the world to work for them (I know one of them, he should do an AMA because it would be super interesting).

The profitable bots are a lot more than just shove bots. They take into account: Hand ranges, Nash Equilibrium (This is really cool stuff imo),EV of flop/turn/river, return from promotions, rake,reaction time of user, position, table image

The last I saw the custom bots being run were buying up hand histories by the millions and using pre-flop agression and all the other factors to preform clustering. Then they clusters were used to choose a strategy.

Stars and FTP are both pretty good about finding bots. But poker bot writers are learning from game bot makers. They are pretty well protected on the client end. So as long as the bots have some variance they are really hard to spot. And a lot of the variance comes from choosing how to play the person who is betting. Though it looks like some bot writers have not been putting enough variance into their bots.

For a long time the big push was to find ways to make the input look like it came from a human. Moving the cursor to the exact middle of the button you want to push every time in <100ms is a dead give away. I don't really know if this was ever something that FTP and stars used though.

Going further, it would be trivial to randomize delays and user input, but the more difficult thing would be to make it random in a human way. Human input is obviously not random but fits into detectable patterns with certain levels of variance. I'm sure the people running the sites could use all sorts of stats in their bot detection algorithm such as when the time gets past midnight in the local time zone human users starts to get sleepy and the avg response time goes down by 50ms or whatever...

> Moving the cursor to the exact middle of the button you want to push every time in <100ms is a dead give away.

This seems intuitive, but it is actually false. The majority of "grinders" (mid-stakes players who multi-table 9-24 tables at a time for a living) use programs like autohotkey that actually do exactly this, allowing players to play with the keyboard and even console (e.g. PS3) controllers. The most popular poker sites explicitly allow those helper programs.

Clicking the button in the exact same spot is not a good way to detect a robot though. Many multi-tabling players use hotkeys to trigger their actions, which is much nicer than a mouse and is almost essential once you get up to 12+ tables.

Also the hotkey software is explicitly allowed by poker sites, as long as no automatic moves are taken (e.g. auto-folding).

AutoIt and others don't actually click the button though. So they are not the same behavior.

Poker sites can tell if the keyboard interrupt came from which usb id so that isn't really an option. Moving the mouse is much more likely not to have a way to see if the hardware caused it. Hence why it is used more often.

I built a bot that used screen scraping, OCR and play reconstruction from the beginning. I had pretty generic logic to handle all the major sites. When a site changed up their display, I just generalized my approach to handle it. I don't really see how they could counter this in general. My bot was never detected. I timed my bot based on statistics I gathered from other human players, including play schedule, breaks, click response time, etc.

I know a guy who worked for an international gambling syndicate, primarily betting on horse races. According to him, they tried to get into online poker bots, but kept getting caught. It sounded like they were using groups of bots and collusion, which I think is how they were detected.

Did you make money?

No. I don't really play poker. I was responsible for the software. My poker guy didn't step up to do the strategy. I thought about learning poker seriously myself, but eventually decided getting a PhD in math was a better use of time, so I went that route.

I'm giving away a piece of data I planned to keep private and use for my own purposes, but here goes. Full Tilt Poker is now fully Qt. While they built their own slightly modified Qt binaries, it's not hard to get QtScript embedded in FTP. Once you have this, you can get the QObject corresponding to the window handle of a game table, and work backwards from the signals/slots there to find the model QObject. From here, you can enumerate all of the methods and events that the table model object exposes, giving you full access to all information that the client has, as well as allowing you to directly control the client.

I built a framework that lets me use QtScript as a bridge from .NET to FTP, enabling me to trivially introspect and manipulate the game client. Never had time to actually hook bot logic up to it, sadly.

Awesome stuff. Though I think it would be pretty hard to use. Last I saw the big sites were all monitoring dll loads and os hooks. So actually doing any actions would bring down their wrath.

It's amazingly easy to get around such things. Eventually I'll write up a blog post on how I've gone about it, and potential future ways of doing it.

For the record, monitoring DLL loads is silly, because well, it only monitors DLL loads. If you load the DLL into memory yourself (essentially replicating the standard functionality), it's a non-issue, and that's not terribly hard.

load the DLL into memory yourself

Does this work with data execution prevention?

Yes, DEP doesn't come into play if you properly set the protection on your code pages. The DLL specifies the protection for each section of the file, so that much is easy.

For the record, i'd be willing to sell all my work to this end, if anyone is interested in botting FTP. Shoot me an email if you're interested.

Thank you, sir, for putting a bit of the "hacker" back into hacker news.

Is there a public framework that allows integration between QtScript/QObject and .NET? If not, you might get a lot of interest if you release that part separately.

No, I built my own. It really doesn't do much: it can turn a QScriptValue (think that's the name -- been a while and I'm on my phone) and turns it into one of a few basic types on the .NET side, and does the inverse. It also let's you call methods on QSVs, or hook up to signals. I just built it as needed to make my FTP hack work. It's got some nice magic that allows you to find connections between QObjects, though, which is how I can get access to everything in the client just from the lobby/table window handle. Qt doesn't provide the neccesary functionality, so my framework patches and hooks a few things, e.g. signal connections.

Edit: to address the separate release part, I really don't think I'd want to do that with this. It's one of the few serious barriers to entry (the object traversal magic, specifically) that makes it non-trivial to bot the hell out of FTP. While I like poker bots and all that, it's the most valuable part of what I have, by far.

I had to do type conversion between QVariant and something similar I designed for an embedded C platform and later included in a Qt GUI. It's definitely the signal/slot traversal I found most interesting though, so I understand the desire not to release it (I'm not trying to do anything with FTP, I'm just interested in language/framework interop in general).

> The suit also accused Full Tilt pros Chris Ferguson and Andy Bloch of creating bots. ... Ferguson became a big tournament winner, with very little tournament experience, because of the edge he had.

The Chris Ferguson? aka "Jesus"? I don't know which time period they are referring to, but he has been one of the most recognizable, well-known tournament players of the last 10 years.

From http://en.wikipedia.org/wiki/Chris_Ferguson

> In 1994, he began playing in tournaments in California and in 1995, he entered his first World Series of Poker. Ferguson beat T. J. Cloutier at the main event of the 2000 WSOP to win the $1.5 million prize.

So he became a winner of the WSOP after first entering 5 years previously. In what way is that "very little tournament experience"? I must be missing something...

WSOP is far from the only tournament; they're implying Ferguson was a tournament winner very soon after first starting to play in them. Do you believe otherwise?

> This score is based on a composite of a variety of factors including the length of their sessions, the standard deviation of those lengths, times played, and consistency of play schedule

It seems that anyone able to write a poker bot is able to tell it to take breaks every now and then. This doesn't seem like a war the poker sites can win.

> anyone able to write a poker bot is able to tell it to take breaks

You can always go after those players who take breaks that are too random ;-)

The players still need to be mindful of how successful they are. It's easy to stand out when you're using illegal methods. Take a look at the plot from this article showing one poker cheater's win rate:

http://simplecomplexity.net/catching-an-online-poker-cheater...

Your article is about actual cheating, not just automated "bot" play

I know. But presumably a bot is programmed to play consistently well ... otherwise there's no point in automating the process. I'm not saying a bot's performance would approach that of someone that's cheating, but if they can be consistently successful to the point where they can play a game unattended, I suspect their dot would fall towards (or beyond) the edges of most other players.

Maybe if it was just playing heads up (1v1), but with technology where it is today, playing better than a group of good humans is apparently a hard problem.

Yes, heads-up limit holdem is basically a solved game from an AI perspective.

The majority of recreational players are really bad. All you have to do is make a bot that is a little bit better and not trivially exploitable. It doesn't matter how good you are, you still lose some. All you have to do to win money in the long run is be "better enough" than the majority of the competition that you can beat the rake.

How would you tell the difference between a bot who plays consistently well and a human who does? It's pretty trivial to legitimately put in 10K hands per day. You think that with 250K hands in a month, a human player doesn't start to play consistently well? At that point, there will be long stretches where the players is basically on auto-pilot, a human-powered robot if you will.

Honestly one difference that comes to mind is that machines don't go on tilt. I knew quite a few people in the community of winning players and I never came across a single one who didn't admit to playing differently/worse while on an extended losing streak. And that includes the 250K hands a month guy. I think for a bot to be undetectable you would have to actually mimic emotional responses to losing money.

I envision a bot as a neural-network-based engine that adjusts its play based on fitting different opponents into different patterns. Honestly, the easiest way to develop a potentially-good bot would be to give it every piece of input and try to evolve a good player, as opposed to trying to develop all the logic manually. Such a bot is guaranteed to have some artifacts which cause it to respond to certain situations and generally play sub-optimally, basically making it look like a human player, anyways.

This is a totally different ball of wax.

This cheater had access to everyone's hole cards and could play perfectly.

The bots in the parent article don't have that access. They're limited to making the same decisions that a regular person could make while playing, they only know about their own cards and the cards on the board.

Keep in mind a bot player would not need to be hugely successful - you can launch hundreds of them that sit just above breakeven.

It's a lot like algorithmic traders, BTW.

But then, how random is too random???

http://img51.imageshack.us/img51/2848/dilberttourofaccountin...

The software mentioned in the article has functionality to take random breaks.

There are plenty of other indicators as well, but then that leads you to kick off the obsessive-compulsive maths prodigies comfortable playing six tournaments at once to grind out an income.

You don't have to be a math prodigy to play six tournaments at once. I used to play six 6-handed cash games at once, and one day I played 16 full ring cash games at a time for like twelve hours straight.

The craziest players can sustain about 50 tables at a time. That is not an exaggeration.

I'm surprised no poker site has launched a site that specifically geared towards algorithmic players. Seems like the kind of thing that might have a niche.

It seems so. However, notice that the average poker bot programmer is a 20-something, highly intelligent hacker and that most of them don't have all that much money. Why do they write poker bots? (a) to make money. (b) to beat the system. (c) boredom.

(a) doesn't work when all the other players are also good poker bots. Botting lives from the simple fact that most poker players are shit. (b) also doesn't apply, for obvious reasons. So, now you are left with the (c)s. Those will leave when they find something more interesting.

Seemed to be a good idea. Well, it's probably not.

(PS: There are poker bot competitions. The bigger ones have university teams competing, the smaller ones (the ones I found via Google) usually don't get enough entrants.)

I guess the stock market is the natural alternative for applying similar skills in a much bigger game.

Betfair allows (i.e has a pubic API) algo players for some of their games, not sure if it covers their Poker site though.

Huh. I'm reading over the docs right now, and while I don't see anything about actually playing the game, they expose an interface for betting on hands. That's pretty interesting.

i actually tried. i couldn't get enough interested parties and had a cofounder that wasn't very motivated.

it would take your game history logs, analyze them, and let you know where your most common mistakes were based on context (i.e., ICM for SNGs)

i still think its a good idea, for various reasons, i've just moved on.

"Good" players, including bot writers want, and need, very bad players in the games. The games need players who either don't know or don't care that they're making sub-optimal plays. That's what makes the money flow in poker. If everyone played perfectly, then only the house would win.

http://www.computerpokercompetition.org/

> At its heart, the Annual Computer Poker Competition aims to benefit artificial intelligence by promoting, aiding, and evaluating research in the challenging problems presented by the wide variety of poker games.

There was a fantastic series of technical articles written a couple of years ago on building a poker bot: http://www.codingthewheel.com/archives/how-i-built-a-working...

The series of articles goes into detail on how to write a successful bot - Statistics, OCR, hooking dlls, etc. There is a ton of information that lays a solid ground work for doing this yourself (if you have the time.)

That link is superb; highly, highly recommended. Can't believe I didn't come across it before. You should submit it separately. I would except you should grab the karma (if you care about it) ;-)

poker sites where bots are welcome http://www.nyse.com/ and http://www.nasdaq.com/

And you get paid whether you win or not.

not only are they welcome, they're the majority, participating in about 80% of the hands dealt

These guys knew they had bot. PokerStars do a good job removing them. Full tilt knew they were there but kind of tolerated them (no idea why). The bot community says (people who made them) they had unwritten agreement of tolerance from FTilt. But now HR2267 passed which give high hopes of getting online poker regulated and all these sites would want a license so they are cleaning up the ship to get in good books. http://www.govtrack.us/congress/bill.xpd?bill=h111-2267

Also look at Lee Rousso case against Washington state which has a law making online poker play a felony. State Supreme court ruled the law as constitutional. Now PokerStars had moved out of Washington state for the same reason.

If it was pre HR 2267 PokerStars wouldn't have done it.

From the title, I was expecting this article to be about how the bots were evolving to play against each other. Sort of like HFT algorithms. Fortunately, it seems like they're still a small enough piece of the population that that's not a significant issue.

Betfair actually provide an API, so that you could easily write a bot for it:

http://bdp.betfair.com/index.php?option=com_content&task...

They provide the API for the following: ‘Exchange Baccarat’, ‘Exchange Blackjack’ ‘Exchange Hilo’ and ‘Exchange Poker’. From reading the site, 'Exchange' seems to mean the games are 25% faster than standard.

Haven't developed anything using their API, but it looks interesting. Also the polar opposite approach of PokerStars and FullTilt.

If there's one thing I'm afraid of, it's the fact that robots are written to expect human input and respond to that input, but in practice, robots also unknowingly receive robot input, which results in weird, unexpected things happening. One example is high-frequency trading robots in the stock market, where one trade (or bug) could set off stuff like flash crashes (http://www.nytimes.com/2010/11/09/business/09flash.html)

Do any of the sites use captcha? That wouldn't stop a human nursing a PC running bots, but it would stop someone starting a bunch of bots, then going to sleep for 8 hours.

argh.. every time I read these articles I wish my wife would let me play poker again.. =( .... I am so sad...

If you can't beat a bot then poker is not the right game for you.

And what if several of them sit at your table and collude?

Would that actually be an advantage? I mean the bots are going to win more often that the other players, but on the other hand, they would also be at least twice as expensive to use.

If you have N seats, you have better odds for the best hand than you do with N-1 seats, yes, but what's even more important than that is that you know what cards the other players don't have. You have considerably more information, and this is absolutely essential. Collusion is the big problem with bots, not just smart logic.

I haven't played Hold 'em since fixed limit died (RIP party poker) but knowing even just two additional cards would be a tremendous advantage. You have an easy fold in small pots if you know you only have 5 outs vs 7 (overcard and gutshot draw). Hell you probably couldn't even be beat in the short run if you knew of 4 additional cards in addition to your hole cards.

Collusion absolutely provides unfair advantages (which is why it's against the rules).

Poker is a game of incomplete information. The more information you possess, the better your odds are of making the right decision.

So, in the instance of 2 poker bots colluding on a table (the 4 suits are (c)lubs (d)iamonds (s)pades (h)earts):

  Bot 1 has Ac5h and discards pre-flop (no money spent)
  Bot 2 has KcQc and calls. 
  Player 3 calls
  Player 4 calls
  All other players discard

  The flop comes up: 5c 10c 9c

At this point, Bot 2 has an excellent hand (a King-high flush), but it doesn't have the best possible hand; it's vulnerable to a higher flush due to the Ace. However, the because bot 1 & 2 have colluded, it knows with 100% certainty that it has an unbeatable hand and can play accordingly.

This is a pretty weak example, but it's the easiest I could think of without assuming any knowledge of poker. Having knowledge of cards that have been in play significantly increases a player's ability to make decisions in a way that give that player an unfair advantage.

> it knows with 100% certainty that it has an unbeatable hand

While your example is a good one, the bot actually doesn't have an unbeatable hand. Another player could have trips or 4/5 of a straight flush and hit a better hand (quads or a straight flush) on the turn/river. Or the board could pair twice and give someone a full house. Unlikely in any case, but not out of the realm of possibilities by any means.

Haha damnit, you're right. Thanks for pointing that out!

anyone with 130k hands would've caught that :)

but seriously, he means if the board is like: 2c 3c 7c Tc 4h, (not that he'd get much value from the guy with the lone Jc, though)

think of how easy squeezing is with two colluding bots.

The problem was that I changed the example after typing everything up. The first contrived example I came up with used a hand that Bot 1 wouldn't have realistically thrown away. My fault for not proof-reading :D

Thanks, it didn't occur to me that you can discard pre-flop.

A more subtle example involves advertising. You want both to remain at the table. One is getting low. Then they both bet into a pot aggressively, everyone else drops out, then the richer one folds and the poor one shows the bluff. Now money has been transferred, and the table thinks one of the bots is a loose player.

There are bots that are capable of beating professional players today. Is poker not the right game for them either?

http://en.wikipedia.org/wiki/Polaris_(poker_bot)

Hands in limit poker nearly always have a mathematically correct way to play; if a program executes the correct play every time and isn't figured out as a bot the program stands a pretty good chance of being profitable.

Most human players don't have the advantage of being able to memorize the correct action for every scenario, a well written bot has an absolute advantage with regards to the math involved.

Additionally, a poker bot remembers every hand it's ever played, as well as every hand it's ever seen played - there has been poker software available for many years that will analyze other player's play for weaknesses which can be trivially exploited. I would assume diligent poker bot creators know about this type of thing and are taking advantage of that as well.

Ok, calm down.

I should've clarified no limit. This article is presumably talking about no limit - since that is by far the predominant game.

With all due respect, you have a poor understanding of poker if you really think there is a way to "memorize the correct action for every scenario". Poker is about interaction, spotaneity, 'feel', instinct. Mastering those things is where to get a real edge, not math - particularly in no limit. Hence my comment.

I'm sure I have a lot of room to improve my game; I've only got around 125k hands logged and haven't played seriously in 4 or 5 years.

That being said, I'm fairly certain the article was not talking about no-limit. The vast majority of poker bots play limit, where there is a clear advantage from playing the mathematically correct way.

The idea that you can play winning poker with "spontaneity" and "feel" is exactly how these bots make money. I'm not going to waste my time trying to explain it to you. If you really care, spend more than 30 seconds reading up about the mechanics of limit poker.

I would suggest starting here, since it has a reasonably compact rundown of the math behind the game:

http://www.amazon.com/Theory-Poker-Professional-Player-Teach...

There is at least as clear an advantage from playing 'mathematically correct' in no limit. It would be a much more difficult bot to write because adding the bet size variable would greatly increase the complexity of the math, but it could also probably outperform human players by a much greater margin for the same reason.

An objection that could be raised is that psychological factors carry more weight in no limit, especially short term psychology (recognizing when a player is on tilt), but I see no reason why a sophisticated program couldn't integrate these factors better than a human. Psychology in online poker is really just pattern analysis.

Wrong, there is a GTO strategy that a bot could employ which is the correct action for every scenario. Actually, it wouldn't be the same action every time for the same scenario, instead randomly bluffing with the correct frequency. You could try to guess all you want but it wouldn't matter. No amount of feel or instinct would allow you to beat it.

Chess used to be like that

Reply on news.ycombinator.com