Memoirs of a Poker Bot Developer
mattmazur.com
mattmazur.com
Both are very clear about the difficulties involved - my suspicion is they push the "eating glass" type analogies primarily so that they don't appear to be encouraging further development. After all, the challenges involved appear to be what motivates them (and many Hackers) in the first place.
My initial impression is that he made some things way more complicated than they had to be and other things way simpler than they needed to be.
But who knows--there's definitely more than one way to do it.
That should only matter if your motivation for developing a poker bot was to develop a program that could demonstrate "skill" rather than one that could produce the highest profits (such as if you were a computer scientist). If you wanted to produce the highest profits, then your metric for comparing implementations would be net profit over the long run.
It is okay to come up with some set of metrics to attempt to measure "skill" in the short term while developing your algorithm, perhaps, but that's when you're in "computer scientist" mode. Ultimately skill is irrelevant if your goal is to make a profit over time, and if it were not, I suspect you would not be working on such a project in the first place.
Still, you can easily determine the mathematical certainty for a given win rate and sample size.
People like to think that if you can build a winning bot you can easily run it on a dozen computers and rake in millions of dollars. In reality, the logistics of running such an operation are prohibitively difficult--at least if you want to stay within the bounds of the law.
Anything specific you want to hear about?
This topic is important and deserves a long discussion--its one of the main things I want to write about in the future.
Once upon a time Party Poker offered a guaranteed 10% APR on all balances up to $10k, paid out monthly given a certain relatively small level of pay. I imagine their multiple account detection process back then was relatively crude compared to what Stars has now, but it caused me some pain. And I know Full Tilt is very good at it now.
The money was a nice, but it really wasn't the primary goal. Plus, in order to maximize profits I would have had to tread in a lot of murky waters that I didn't want to be near.
For example, say you have a roommate run the bot. How do you get the profits from his poker account into your bank account? Not as easy as it might seem.
But in those states, perhaps. I never had to pay an attorney to review that :)
Those are ancillary factors that have an effect on net profit. They are not metrics. Taking the story from the post, if my "friends and family" were asking me about the "success" of a poker bot, I'd still point to net profit as the best metric with which to define success (unless you're doing it for CS/math reasons).
It's a bit like if I'm a professional, regular poker player and family members ask me how I'm doing. I'd tell them how much money I've won (or lost), not the intricate details of how I play.
We're obsessed with the performance metric, big bets per hour for live players, big bets per 100 hands (BB/100) for online. Standard deviations, results ranges, levels of certainty. You obsess about that stuff or you end up broke. (Of course, you may do both.)
In fact if you read online poker forums, hands and win rates are rarely ever discussed with dollar values because they aren't very relevant. They're almost always given in terms of big bets, and most if not all online hand converters (which take a hand history from a poker site and print it out in an easy to read format for posting on forums) convert to that automatically.
The absolute dollar amounts are only useful for getting a very rough estimate of the level of competition given no other information.
So are you saying that, ultimately, profit is the most important metric? ;-)
If only my poker pro friends were as considerate.
Seriously though, in the long run the money is the metric. Holding your own against great players won't pay the bills. Finding weak players at decent stakes is where it is at and as such this is part of the game.
They are masters of deception and even if the post is tech related it should be watched closely.
I know them for I've been inside the beast.