How I built a poker-bot
codingthewheel.com
codingthewheel.com
If your bot can play 14 limit tables at a time with even the slightest % edge, you're going to make money. However, if you added more advanced player profiling algorithms into your bot, you're edge would likely increase and allow you to set it up at higher stakes.
You have to leave the table as soon as you get past 1.5x the minimum buy-in though. It was mainly done on Party Poker, which was the only site dumb enough to let you cash out and then return to the table for the min again. Any other site either made you bring back what you left with or wait an hour.
On a site with enough tables in action, though, even that would not be a limitation. A bot could do fairly well at it.
Also I can tell you from first hand experience that breaking even at low stakes NL sit and goes can be done with an AI consisting of nothing but if/then statments. I did this with WinHoldem back in the days before it was banned from everywhere. I wasn't able to replicate that success with limit. The main problem with WinHoldem was that it didn't join or leave tables on its own (also, it would often bug out when the tournament got heads up), which meant that the time spent managing it made it less profitable than just playing myself.
That's why we tried to build a bot ourselves. Maybe I'll blog about that some time.
While I will agree sit and gos are all strategy that strategy is dependent on putting your opponent on reasonable ranges which a bot like this won't do.
Bots and winning bots are possible(stick most casual people up against the UofA bots for example), but I find it hard to believe this one is one of them.
If you went to a no-limit tournament and just pushed all of your chips into the middle at every single opportunity, it's been shown that you're only a slight loser. At the WSOP main event, you'd be a better player than at least 1/3 of the field.
This is not true if you were to play in a limit tournament and raise every time. You'd probably be the worst player in the field.
The fact that it's less complicated is why no-limit is the tournament game of choice, but high stakes live poker is generally limit.
Pushing every hand isn't right, it's pushing every hand with certain cards that Sklansky taught which admittedly is only a slight loser but a loser none the less.
I'm not saying limit holdem isn't a complex deep game, all I'm saying is that when it comes to bots no-limit bots are a lot harder to get right(in opponent modelling primarily which is a big part of bots these days).
I think no-limit is the tournament game of choice because it's seen by the general public as being the more exciting of the two(not that I agree but thats another discussion).
But if you're building from the standpoint of breaking even at low stakes, it's far easier to do for no-limit, especially sit and goes. It's literally a couple hundred lines of if/then statements.
Just because there is a simple strategy (shortstacking push/fold) that can be applied successfully against weak players, that doesn't mean that NL is a simpler game.
When you are trying to model the game there is a lot more to consider in NL. In Limit at each point in a hand and at each future point in a hand there are only limited actions that your opponent can take - raise, call, check, fold. In NL they can do any of these actions but also for any amount - push, raise pot, raise 1/2 pot etc. So the problem space for NL is much larger than for Limit. Bluffing is a much bigger part of NL and bluffing is hard to model.
Playing profitable limit poker is a fairly mechanical, mathematical process. You can't force people out of a hand with a big bet. It basically boils down to betting when you're probably ahead, folding when you're probably behind and drawing when the pot odds are favorable.
The U of A bots can play limit reasonably successfully against decent players. A decent player can easily beat their bots at NL.
Building a bot that can profit at the microstakes tables might be easier for NL than limit because you can employ a push/fold short stacking strategy like you describe. But building a bot that can survive against better players would be much harder for NL than Limit.
I'm not so sure about that. I think the main reason is: there's a lot more "stupid money" at the poker table than there is in the market.
The bot as mentioned doesn't seem to do anything particularly fancy -- it just looks at the cards on the table and figures out your odds of winning based on that. This is information which is easily extracted mathematically, but which most players don't have direct access to. Good players will have a pretty good idea of these probabilities, whereas lousy players won't. If you can find yourself on a table full of lousy players, you're at a huge advantage.
In the market, on the other hand, there's only one poker table, and all the best players in the world are already sitting at it. Any time statistical arbitrage opportunities arise in prices, you can bet that D.E. Shaw and Rennaisance Technologies have already noticed it and moved a few billion dollars that way. If you want to win, you don't just have to be smarter than six guys in their basements doing calculations in their heads, you have to be smarter than thousands of overpaid quants with supercomputers.
Anyway, the article, although it describes well the techniques for programmatically interacting with a poker client, is majorly misleading because this is not the hard part of making a bot. The hard part is figuring out whether, and how much, to bet. He alludes to this as "the million dollar question", but then omits to even mention it under "What skills will I need to write a bot?" Perhaps he'll address this in later articles?
If you look at the last screen shot, it's clear that he built a UI for an end-user to specify the actual logic for poker play. ("Call if it has not been raised, I'm later than 7th", etc.) His "bot" doesn't actually make decisions. It collects rules from an end-user, calls those to decide what to do, and passes the result on to the poker client. In other words, the hard part of the job - writing a program to play poker - has been passed to somebody else. Maybe that somebody else is the author, creating the algorithm through his UI... but then why not just code it?
There are serious researchers who have made credible bots (he links to them), bots which would (and, for all we know, do) make money on poker sites. But when I looked at it, the literature was not nearly detailed enough to code up their algorithms. (Also, they were working with simplified versions of poker, since full no-limit holdem is a hard nut to crack). What they didn't use were the "extensible rules-based systems" which this author claims "are a lot more powerful than you think". Count me skeptical. Good players eat such systems for snacks. Maybe they make money in low-stakes games against bad players, but I'd like to see evidence.
In short: never mind the dll injection, show us your bankroll over 100,000 hands!
You're thinking about it wrong. As he mentions, your bot doesn't play against good players. It plays .5/1 or 1/2 nl and makes a shit ton of money if it just breaks even. There aren't good players at those levels, because anyone who was good would win and quickly move up. You only need all of your if/then statements to add up to being better than the worst 5% of poker players, and that's not that hard to do. I know only because I've done it.
Thats why extensible rules based systems are better than you think. They can get you to that with a good amount of work. Especially in short stack nl games. They're clearly not going to win at a $5/$10 nl table or a $215 SNG. But they don't need to to rake in millions per year.
Lokibot: http://www.cs.ualberta.ca/~jonathan/Grad/papp/thesis.html
The bot described in that paper lost about 8c a hand at the $1 tables. After extensive modifications, we got it to the point where it was losing about 2c a hand. This when Bush made poker illegal in the US, and we killed the project.
That's a tough choice for a casino. Allow automation and you create a ghetto which discourages casual users. Disallow automation and you encourage an escalating "arms race".
Regardless, the techniques discussed could be detected using anti-virus techniques. I'm surprised that native clients don't perform integrity checks or is this the state of play in the "arms race"?
It would be fairly simple to disable integrity checks using methods to circumvent software DRM. Then it would be a matter of implementing distributed state to determine and ignore circumvented clients.
It doesn't feel pity, or remorse, or fear. And it absolutely will not stop, ever, until UltimateBet is rich.
So, it is in a casino's interest to leave the whole issue unresolved and with a level of tolerance which is in-step with competitors.
And as a second crucial step, you hook groups of bots into a "monitoring computer", at which an actual human sits and responds "No, I'm not a bot lol." when the supposedly fatal question arrives from an admin. You can even outsource this to the third world.
Just the computer vision aspect of it alone would be amazingly hard.
Efficient goldfarming (e.g., with a frost mage that can take on 15-30 monsters at once) involves carefully coordinating a complicated series of actions where timing is critical and reacting quickly and intelligently when things go wrong is the difference between living and dying. It involves making complicated judgments about relative positioning, reasoning about surroundings, cause and effect, and other things that require human-level intelligence.
If you can make a bot that can seek out and kill monsters in a non-trivial manner without dying all the time there's no reason you couldn't program a robot to replace a wide variety of menial real-world jobs. The consequences would be tremendous and using it to make $20/day off the Internet would be retarded because you could license the technology for billions.
Although I do agree that with that technology you are better off licensing it.
Check out http://www.easyuo.com. That application allows you to easily automate Ultima Online. Whilst deciphering WoW's application memory would probably be harder, an equivalent program is certainly possible (it probably already partially exists in various forms), and intelligent bots are a simple step after that.
Unfortunately, once the memory accessing has been abstracted away, programming a competent WoW bot is not a difficult task. The WoW bot doesn't have to be as awesome as the best human players, nor need it be ultra-efficient. What it lacks in ability it makes up in working without fatigue indefinitely.
I'm pretty sure WoW keeps track of all applications running on your computer. I think the way people do it is by using a hub and packet sniffers on a separate computer, although I imagine WoW traffic could easily be encrypted... so shrug.
A bit like the matrix.
And if randomness is a dead giveaway of a bot, then don't implement it?... I was suggesting to add it but if it actually ampers your goals then make a fully deterministic bot...
I wonder mattmaroon's thoughts on this.