http://freakonomics.blogs.nytimes.com/2008/07/14/overreacting-to-a-computer-beating-poker-pros/

It’s not just luck; there are important decisions to make all the time (unlike roulette, craps, etc). But it’s true that luck plays a big part, which is why poker players tend to measure success in terms of how well they do over the course of a year, say.

http://hardware.slashdot.org/article.pl?sid=08/07/09/2014228

s/under/over/ Or maybe just “misestimating”. It’s very easy to generalize incorrectly in either direction from this story. In reality, it’s no more (nor less, to be sure) impressive than, say, a medical or financial system with a few hundred variables and a few dozens of models to weigh.

http://poker.cs.ualberta.ca/papers/Papers/johanson.msc.pdf is a good semi-technical description of how the software works. As you read it, you’re likely to be less impressed with it as a measure of “machine ability”, and perhaps more impressed with the engineering and theory that goes into “machine-creating ability”.

The linked article indicates this is not the case.

“And secondly, we have added an element of learning, where Polaris identifies which common poker stratagy a human is using and switches its own strategy to counter. This complicated the human players ability to compare notes, since Polaris chose a different strategy to use against each of the humans it played,” Bowling said.”

…

Before the Las Vegas match, this newest version of Polaris had only played two matches against champion poker players, resulting in one loss and one victory. Polaris repeated the pattern of improving as it learned, falling to humans in the first two rounds, but defeating them in rounds three and four. “Repeatedly, I heard players exclaim that they had never seen a human do that before,” said Bowling. “Switching strategies really threw the humans for a loop.”

This was no decisive blow, it was a single test, where each round consisted of only 500 hands of one of the simplest forms of poker, and the AI come out on top by a rather small margin. Could very well have been luck. Further tests will tell.

That said, I will not be terribly surprised even if we get real evidence that AIs can now beat humans at Heads-Up Limit Hold’em. My guess however is that this current AI will not end up demonstrating decisive superiority, but that we might very well see such a superior AI in 1-3 years. Or not.

And that is if we are talking about Limit poker. In the case of No Limit, I find it very hard to guess when AIs will reach the human level of ability. Won’t be extremely soon, and the problem might even be AI-Complete, requiring essentially human-level ability to model the strategies of another human. No Limit poker is not so much straightforward math as Limit poker is. It is more about being better at modeling your opponent’s decision-making than they are at modeling yours. (A nice shortcut to seeing whether this Polaris AI will eventually end up easily beatable by some pros would actually be to hand over it’s source code for their examination. Even Limit poker might be substantially less mathy at the highest level than I estimate. I am not a Limit pro.)

By the way, AI bots have already been used for several years to make lots of money playing Limit poker online. Some people have farms of bots playing lots of tables. The poker sites disallow and (some) try to monitor this, but there are always anti-detection solutions. The prevalence of bots in Limit poker has caused a lot of people to shift over to playing No Limit.

In other words, this poker bot, if it’s the one I’ve heard of before, doesn’t play to win. It plays not to lose, using no opponent modeling whatsoever. If its opponent doesn’t make “dominated errors” – moves that are strictly worse than another – it will tend to break even, no matter how predictable its opponent is. Furthermore, a move is a dominated error if and only if an optimal strategy uses it with probability zero. Even if the optimal strategy says “make this move with probability 1/1000″ then it won’t do better than break even against a strategy that says “make this move with probability 1.” However, Poker is sufficiently complicated that it’s not apparent whether a move could be a dominated error, so there might still be ways for a human to screw up that an “optimal” bot will exploit.

(Also, the bot uses an approximation to the optimal strategy, as finding the true optimal strategy is computationally intractable. However, it does seem to be a pretty good approximation, as it has been successful against top human players.)