A computer has beaten top human poker players: Humanity was dealt a decisive blow by a poker-playing artificial intelligence program called Polaris during the Man-Machine Poker Competition in Las Vegas. Poker champs fought the AI system to a draw, then won in the first two of four rounds (each round had Polaris playing 500 hands against two humans, whose points were averaged.) But in the final two rounds of the match, Polaris beat both human teams, two wins out of four, with one loss and one draw. IBM’s Deep Blue beat chess champion
The set of those people who are still reading this thread and who do not check in on the Freakonomics blog is probably pretty slim, but for those in that intersection, you might want to check out Levitt's two cents:
in poker there is a lot of luck involved, isn't it? Then how a computer could excel in poker games?
It's not just luck; there are important decisions to make all the time (unlike roulette, craps, etc). But it's true that luck plays a big part, which is why poker players tend to measure success in terms of how well they do over the course of a year, say.
The moral: beware of underestimating computers by underestimating the difficulty of the tasks at which they excel.
s/under/over/ Or maybe just "misestimating". It's very easy to generalize incorrectly in either direction from this story. In reality, it's no more (nor less, to be sure) impressive than, say, a medical or financial system with a few hundred variables and a few dozens of models to weigh.
http://poker.cs.ualberta.ca... is a good semi-technical description of how the software works. As you read it, you're likely to be less impressed with it as a measure of "machine ability", and perhaps more impressed with the engineering and theory that goes into "machine-creating ability".
Doug S:It plays not to lose, using no opponent modeling whatsoever.
The linked article indicates this is not the case.
"And secondly, we have added an element of learning, where Polaris identifies which common poker stratagy a human is using and switches its own strategy to counter. This complicated the human players ability to compare notes, since Polaris chose a different strategy to use against each of the humans it played," Bowling said."...Before the Las Vegas match, this newest version of Polaris had only played two matches against champion poker players, resulting in one loss and one victory. Polaris repeated the pattern of improving as it learned, falling to humans in the first two rounds, but defeating them in rounds three and four. "Repeatedly, I heard players exclaim that they had never seen a human do that before," said Bowling. "Switching strategies really threw the humans for a loop."
As a semi-professional poker player, I'd like to say that the article being quited does not understand the event being reported very well, since they say "Humanity was dealt a decisive blow by a poker-playing artificial intelligence program".
This was no decisive blow, it was a single test, where each round consisted of only 500 hands of one of the simplest forms of poker, and the AI come out on top by a rather small margin. Could very well have been luck. Further tests will tell.
That said, I will not be terribly surprised even if we get real evidence that AIs can now beat humans at Heads-Up Limit Hold'em. My guess however is that this current AI will not end up demonstrating decisive superiority, but that we might very well see such a superior AI in 1-3 years. Or not.
And that is if we are talking about Limit poker. In the case of No Limit, I find it very hard to guess when AIs will reach the human level of ability. Won't be extremely soon, and the problem might even be AI-Complete, requiring essentially human-level ability to model the strategies of another human. No Limit poker is not so much straightforward math as Limit poker is. It is more about being better at modeling your opponent's decision-making than they are at modeling yours. (A nice shortcut to seeing whether this Polaris AI will eventually end up easily beatable by some pros would actually be to hand over it's source code for their examination. Even Limit poker might be substantially less mathy at the highest level than I estimate. I am not a Limit pro.)
By the way, AI bots have already been used for several years to make lots of money playing Limit poker online. Some people have farms of bots playing lots of tables. The poker sites disallow and (some) try to monitor this, but there are always anti-detection solutions. The prevalence of bots in Limit poker has caused a lot of people to shift over to playing No Limit.
The poker-playing AI that I heard about a while ago, which gave a human champion a run for his money, used an approximation to the game-theoretic optimum strategy for heads-up limit poker. To clarify, the game-theoretic optimum strategy the is one that is not exploitable; the most any opponent can do against it is break even. For example, in Rock-Paper-Scissors, the "optimum" strategy is to choose randomly, with an equal probability of making each throw. This strategy cannot be exploited. On the other hand, this strategy is also incapable of exploiting an opponent's mistakes; even an opponent that always chooses Rock will break even.
In other words, this poker bot, if it's the one I've heard of before, doesn't play to win. It plays not to lose, using no opponent modeling whatsoever. If its opponent doesn't make "dominated errors" - moves that are strictly worse than another - it will tend to break even, no matter how predictable its opponent is. Furthermore, a move is a dominated error if and only if an optimal strategy uses it with probability zero. Even if the optimal strategy says "make this move with probability 1/1000" then it won't do better than break even against a strategy that says "make this move with probability 1." However, Poker is sufficiently complicated that it's not apparent whether a move could be a dominated error, so there might still be ways for a human to screw up that an "optimal" bot will exploit.
(Also, the bot uses an approximation to the optimal strategy, as finding the true optimal strategy is computationally intractable. However, it does seem to be a pretty good approximation, as it has been successful against top human players.)
There is a very big gap between the "top human poker players" that Polaris has beaten to the actual top human poker players. Stoxtrader would be absolutely crushed by people like Cole South and Phil Galfond.
The ability to play chess well is widely considered to be a sign of intelligence. Poker does not have the same cachet in the popular mind.
The media would not have paid much attention if a machine had become a top Go player, either, because their audience would know of little reason to regard it as significant.
I think Robin's explanation for why people fail to appreciate the skill of a poker champion is closest. The big difference is the randomness, so even a skilled poker player is going to lose a lot of hands that could have been won with different (and worse!) play. A random spectator has many chances to say, "ha! I would have won that hand." Of course those hands are outweighed by the cases where the spectator would have lost more money than the expert, but good ol' confirmation bias takes care of that.
Also, if you watch TV poker, you get to see the other players' hands, so when you see a player making a losing move, it looks like bad play, when of course it may very well be the correct play.
(If you haven't tried writing a poker AI, you'd be amazed at the complexity of the problem. Take a ridiculously simplified version of poker: two players each draw a card; player one has a blind bet of one chip; player two can fold, call, or raise one chip; player one can call or fold; high card wins. Even that game has remarkable subtleties in the optimal strategy.)
I don't know how seriously to take it, but there's been a recurrent series on building a poker bot on reddit for the last few months:http://www.codingthewheel.c....
From what I've read, detecting physical "tells" is vastly overrated. Detecting tendencies can be invaluable, and already serious online players use software that accumulates stats on all their online opponents.
That was how the cheating at Absolute Poker was detected. People within the company were using accounts that allowed them to see everyone's hole cards. Winning (cheating) players seemed ridiculously lucky to some opponents. They were caught out as impossibly lucky by datamining software.
Those with computer assistance already have a distinct edge. I would think eventually those players with a human component will have a distinct disadvantage.
At the very highest level of play, I would guess that most of the edge that human players have would be washed out by the odds. Why shouldn't an algorithm that does nothing but play the Kelly Criterion win at this level? It's the theory of runs.
CHESS vs. POKER, LIFE & “End Game Theory”]
=================================================================
(1of 4).Difference between,: Playing-Chess, Living-Chess?
(2of 4). Chess-Pieces: King, Queen, Bishop, Knight?,{?}, {? } [which can be
considered the ROOT, in the game of, LIFE & PROFIT?
(3 of 4). Can Chess be performed without a CHESS-BOARD-FORMAT, has it
also been represented/called byanother name?
Justaskingpot-p-s? abnjd
(4 of 4). Can the Origin of theChess Board’s Format & Intent in the 21
Centennial, can be referenced in relation to ,“Common-Sense”, more
Important,as well as a “GAME”?
justasking pot-p-s? abnjd
The set of those people who are still reading this thread and who do not check in on the Freakonomics blog is probably pretty slim, but for those in that intersection, you might want to check out Levitt's two cents:
http://freakonomics.blogs.n...
in poker there is a lot of luck involved, isn't it? Then how a computer could excel in poker games?
It's not just luck; there are important decisions to make all the time (unlike roulette, craps, etc). But it's true that luck plays a big part, which is why poker players tend to measure success in terms of how well they do over the course of a year, say.
poker: the computer can simulate the life cycles of 6 million four-leaf clovers per second, making it luckier than the luckiest human.
in poker there is a lot of luck involved, isn't it? Then how a computer could excel in poker games?
Re: very little news coverage - the story was mentioned on Slashdot:
http://hardware.slashdot.or...
Now that the University of Alberta has a robot that can beat the best poker players I think we can start to see the Edmonton campus expand.
The moral: beware of underestimating computers by underestimating the difficulty of the tasks at which they excel.
s/under/over/ Or maybe just "misestimating". It's very easy to generalize incorrectly in either direction from this story. In reality, it's no more (nor less, to be sure) impressive than, say, a medical or financial system with a few hundred variables and a few dozens of models to weigh.
http://poker.cs.ualberta.ca... is a good semi-technical description of how the software works. As you read it, you're likely to be less impressed with it as a measure of "machine ability", and perhaps more impressed with the engineering and theory that goes into "machine-creating ability".
Doug S:It plays not to lose, using no opponent modeling whatsoever.
The linked article indicates this is not the case.
"And secondly, we have added an element of learning, where Polaris identifies which common poker stratagy a human is using and switches its own strategy to counter. This complicated the human players ability to compare notes, since Polaris chose a different strategy to use against each of the humans it played," Bowling said."...Before the Las Vegas match, this newest version of Polaris had only played two matches against champion poker players, resulting in one loss and one victory. Polaris repeated the pattern of improving as it learned, falling to humans in the first two rounds, but defeating them in rounds three and four. "Repeatedly, I heard players exclaim that they had never seen a human do that before," said Bowling. "Switching strategies really threw the humans for a loop."
As a semi-professional poker player, I'd like to say that the article being quited does not understand the event being reported very well, since they say "Humanity was dealt a decisive blow by a poker-playing artificial intelligence program".
This was no decisive blow, it was a single test, where each round consisted of only 500 hands of one of the simplest forms of poker, and the AI come out on top by a rather small margin. Could very well have been luck. Further tests will tell.
That said, I will not be terribly surprised even if we get real evidence that AIs can now beat humans at Heads-Up Limit Hold'em. My guess however is that this current AI will not end up demonstrating decisive superiority, but that we might very well see such a superior AI in 1-3 years. Or not.
And that is if we are talking about Limit poker. In the case of No Limit, I find it very hard to guess when AIs will reach the human level of ability. Won't be extremely soon, and the problem might even be AI-Complete, requiring essentially human-level ability to model the strategies of another human. No Limit poker is not so much straightforward math as Limit poker is. It is more about being better at modeling your opponent's decision-making than they are at modeling yours. (A nice shortcut to seeing whether this Polaris AI will eventually end up easily beatable by some pros would actually be to hand over it's source code for their examination. Even Limit poker might be substantially less mathy at the highest level than I estimate. I am not a Limit pro.)
By the way, AI bots have already been used for several years to make lots of money playing Limit poker online. Some people have farms of bots playing lots of tables. The poker sites disallow and (some) try to monitor this, but there are always anti-detection solutions. The prevalence of bots in Limit poker has caused a lot of people to shift over to playing No Limit.
The poker-playing AI that I heard about a while ago, which gave a human champion a run for his money, used an approximation to the game-theoretic optimum strategy for heads-up limit poker. To clarify, the game-theoretic optimum strategy the is one that is not exploitable; the most any opponent can do against it is break even. For example, in Rock-Paper-Scissors, the "optimum" strategy is to choose randomly, with an equal probability of making each throw. This strategy cannot be exploited. On the other hand, this strategy is also incapable of exploiting an opponent's mistakes; even an opponent that always chooses Rock will break even.
In other words, this poker bot, if it's the one I've heard of before, doesn't play to win. It plays not to lose, using no opponent modeling whatsoever. If its opponent doesn't make "dominated errors" - moves that are strictly worse than another - it will tend to break even, no matter how predictable its opponent is. Furthermore, a move is a dominated error if and only if an optimal strategy uses it with probability zero. Even if the optimal strategy says "make this move with probability 1/1000" then it won't do better than break even against a strategy that says "make this move with probability 1." However, Poker is sufficiently complicated that it's not apparent whether a move could be a dominated error, so there might still be ways for a human to screw up that an "optimal" bot will exploit.
(Also, the bot uses an approximation to the optimal strategy, as finding the true optimal strategy is computationally intractable. However, it does seem to be a pretty good approximation, as it has been successful against top human players.)
There is a very big gap between the "top human poker players" that Polaris has beaten to the actual top human poker players. Stoxtrader would be absolutely crushed by people like Cole South and Phil Galfond.
This has gained very little news coverage
The ability to play chess well is widely considered to be a sign of intelligence. Poker does not have the same cachet in the popular mind.
The media would not have paid much attention if a machine had become a top Go player, either, because their audience would know of little reason to regard it as significant.
I think Robin's explanation for why people fail to appreciate the skill of a poker champion is closest. The big difference is the randomness, so even a skilled poker player is going to lose a lot of hands that could have been won with different (and worse!) play. A random spectator has many chances to say, "ha! I would have won that hand." Of course those hands are outweighed by the cases where the spectator would have lost more money than the expert, but good ol' confirmation bias takes care of that.
Also, if you watch TV poker, you get to see the other players' hands, so when you see a player making a losing move, it looks like bad play, when of course it may very well be the correct play.
(If you haven't tried writing a poker AI, you'd be amazed at the complexity of the problem. Take a ridiculously simplified version of poker: two players each draw a card; player one has a blind bet of one chip; player two can fold, call, or raise one chip; player one can call or fold; high card wins. Even that game has remarkable subtleties in the optimal strategy.)
--Jeff
I don't know how seriously to take it, but there's been a recurrent series on building a poker bot on reddit for the last few months:http://www.codingthewheel.c....
From what I've read, detecting physical "tells" is vastly overrated. Detecting tendencies can be invaluable, and already serious online players use software that accumulates stats on all their online opponents.
That was how the cheating at Absolute Poker was detected. People within the company were using accounts that allowed them to see everyone's hole cards. Winning (cheating) players seemed ridiculously lucky to some opponents. They were caught out as impossibly lucky by datamining software.
Those with computer assistance already have a distinct edge. I would think eventually those players with a human component will have a distinct disadvantage.
At the very highest level of play, I would guess that most of the edge that human players have would be washed out by the odds. Why shouldn't an algorithm that does nothing but play the Kelly Criterion win at this level? It's the theory of runs.