The poker machine

TIM HARFORD – The Financial Times

Copyright 2006 The Financial Times
May 6, 2006 Saturday
HEADLINE: This man is a computer scientist and an expert in game theory. He is also one of the 21st century’s most successful poker players, and when the game’s World Series starts in Las Vegas next month, everyone will be watching to see if Chris Ferguson can once again use maths to win the title. Are the geeks finally ready to conquer poker?
The World Series of Poker in Las Vegas in 2000 attracted a record 500 players. Over four days, contestants were gradually eliminated until just two men were left to face off in poker’s flagship game, Texas Hold ‘Em. The more experienced player was a living legend named T.J. Cloutier, a 62-year-old Texan road gambler who was regarded by many as the best in the world. His opponent was a 37- year-old computer scientist from California named Chris Ferguson who had only been playing World Series games since 1996, never finishing higher than fourth place.
Ferguson might have been a relative newcomer, but he was hard to miss. He had earned the nickname “Jesus” because he hid his face behind a long beard and hair that cascaded over his shoulders, buttressed by wraparound mirror shades and a big cowboy hat.
Ferguson never spoke during a game, determined not to show any sign of human emotion; he didn’t pay much attention to other players’ nervous tics either, preferring to draw all his information from the cards. In Las Vegas that week he had destroyed the field and came to the table with 10 times as many chips as his opponent.
Cloutier, a former football pro with huge shoulders, paws that dwarfed his cards, and a dominant presence at the table, had seen it all before. Playing brilliantly and riding his luck, he ate into Ferguson’s lead and was only slightly behind when he lured Ferguson into serious trouble. By making a modest raise of Dollars 175,000, he provoked Ferguson into raising again to Dollars 600,000. Then Cloutier pushed about Dollars 2m dollars in chips into the pot, going “all in”.
Ferguson paused, calculating the odds. Cloutier probably had a stronger hand than he’d expected. However, Cloutier was playing well, and if Ferguson backed out of the pot now his opponent would have a substantial lead.
On the other hand, if Ferguson called and won, the World Series was his. He reckoned his chances at about one in three, and that that was as good as it was likely to get.
In a game of poker, players bet to earn the right to compare cards at the end of each hand, the “showdown”. A player who does not bet must drop out of the hand. The accumulated bets make up the prize for each showdown, the “pot”. The best combination of five cards wins the pot.
Strong hands, such as a straight (five cards in sequence) or a full house (a pair and a triplet) are enviable, but also the gift of pure chance. The skill comes in betting and reading other players’ bets. Big bets may scare away an opponent who does not realise he holds the best hand, or they may simply ensure that when your full house wins, the pot is a big one.
In Texas Hold ‘Em, each player holds just two cards. The other three are selected from shared cards, dealt on the table in stages to allow extra rounds of betting. Before the shared cards are dealt, players can only guess how happily their two private cards will fit with the cards on the table. Nevertheless, it is common to see huge bets at that stage – none larger than those of Cloutier and Ferguson.
Finally, after several minutes of furious thought, Ferguson removed his hat and shades, suddenly shrinking and revealing his human qualities of exhaustion and vulnerability – “much more like Jesus”, observed writer James McManus in Positively Fifth Street, his book on playing in the same tournament. (McManus himself was knocked out of the tournament in fifth place.) Then Ferguson called.
Cloutier revealed he had an ace and a queen to Ferguson’s ace and nine. Since there were no more chips to bet, the five communal cards were revealed at an agonising pace. Ferguson had slightly overestimated his chances: they were one in four. He needed a nine to appear on the table to pair the one in his hand, and for no queen to show up and pair Cloutier’s. When the last card – a nine – hit the table, Ferguson realised what had happened before the hushed crowd did. His arms shot into the air and he leapt up to give Cloutier a boney embrace. The great man took the loss with equanimity: “That’s poker,” he said, according to McManus in Positively Fifth Street.
Ferguson’s record since then has proved that the upset that year was no fluke. Only four men have more World Series finishes in profit – poker’s equivalent of making the cut in golf – than Ferguson, and Ferguson won more World Series events from 2000-2004 than any of his rivals had in a decade. So “Jesus” has a respectable claim to be the most successful tournament player of the 21st century, and his ascendancy is more than just a personal triumph: it marks a turning point in the long campaign to apply the rigours of mathematics to the psychological subtleties of poker.
Ferguson, who is reported to have won more than Dollars 5m in tournament play, is the best of a new generation of players trying to conquer poker with the branch of mathematics known as game theory. It is a curious struggle, one that has pitted bespectacled geeks against hardened gamblers. But the strangest thing is that poker intellectuals exist at all.
Late in the 1920s, the most brilliant man in the world decided to work out the correct way to play poker. John von Neumann, the mathematician who would mastermind the development of the computer and the atomic bomb, had been struck by an engaging new conceit: he wanted to apply mathematical principles to social sciences and devise a theory to analyse everything from the breakdown of diplomatic negotiations to unexpected co-operation between enemies, or even the possibility of nuclear terrorism. He believed that if you wanted a theory that could explain life, you should start with a theory that could explain poker – game theory. “Real life consists of bluffing, of little tactics of deception, of asking yourself what is the other man going to think I mean to do, and that is what games are about in my theory.”
Bluff, deception and mind-reading are unpromising subjects for a mathematician to study, but if anyone could do it, it was von Neumann. His biographer, William Poundstone, says his ostentatious feats of calculation were notorious. At Princeton after the war – where he arguably outshone his campus colleague, Albert Einstein – von Neumann helped to design the fastest computer in the world, then demonstrated that he was faster. Nobody was surprised. His colleagues joked that he was a demigod who, having studied humans intensively, was able to imitate them perfectly.
To tackle poker, von Neumann had to break new ground. There had long been analyses of games of chance using probability theory. Then there are games such as chess, which are strictly logical – albeit demanding extraordinary feats of calculation. Poker is another matter altogether.
Most of the important information in poker is private; each player sees only one part of the jigsaw and must piece together the bigger picture by observing what other players do. Since the strongest hand takes all the money, the higher the betting, the more expensive it becomes to lose. Yet in many hands of poker, especially between skilled players, there is no showdown: one player bets aggressively enough to scare the others away. In short, there is no straightforward connection between what a player bets and the hand he holds.
It was the bluff that interested von Neumann. Novices wrongly believe that bluffing is merely a way to win pots with bad cards. In the 1972 final of the World Series, the famous hustler Amarillo Slim won because he had bluffed so often that when he finally put all his chips in the pot with a full house (a very strong hand), his opponent assumed Slim was bluffing again; called (matching the bet), and lost. A player who never bluffs will never win a big pot, because on the rare occasions that he raises the betting, everyone else will fold before committing much money.
Then there’s the reverse bluff: acting weak when you are strong. In the 1988 World Series, the Chinese-born Johnny Chan (dubbed the “Orient Express” because he won money so quickly) passed up every opportunity to raise the stakes and meekly called his opponent’s bets. By the last round of betting, his opponent became convinced that Chan didn’t have a hand and bet everything he had. Chan called and turned over a straight – a strong hand – scooping up Dollars 700,000 and the title of world champion.
These stories seem to be all about psychology, not mathematics. But although Chan and Slim had no interest in mathematics beyond calculating the odds, von Neumann’s theory could explain everything they had done. What von Neumann showed in his ground-breaking 1944 book, A Theory of Games and Economic Behavior (written with economist Oskar Morgenstern), was that you should bluff only with your worst hands, rather than with something half-decent.
The reasoning is simple enough. A modest hand may beat another modest hand, so it’s worth limping along to a low-stakes showdown. A bad hand will only win anything if the opponent folds, so bad hands should be played aggressively or not at all; indeed, when the best players are caught running a huge bluff they are often holding the most atrocious cards. Von Neumann’s model also highlights the other benefit of bluffing: it forces the opponent to match your bids frequently, and so wins more money with strong hands.
Von Neumann’s book was hugely celebrated, but academics were soon disillusioned: they found that game theory was too narrow and too difficult to apply to the real world. The book sold poorly, although a few copies, as the Princeton University Press noted sheepishly in 1949, “were bought by professional gamblers”.
It is a safe bet that young Walter Clyde Pearson was not a customer. “Puggy” Pearson was typical of the hard-living, cigar- chomping, poker professional of his time. He was born in Kentucky in 1929, to a dirt-poor family. But, as Michael Kaplan and Brad Reagan reveal in their book Aces and Kings, when Pearson joined the Navy in 1946 he began to clean up at poker and pool – during an 18- month tour of duty in Puerto Rico he wired home Dollars 10,000 to his mother.
Pearson, who became world champion in 1973, had no use for mathematics. He was an intuitive poker player, and a highly aggressive one, whose big bets often scared other players into folding superior hands. It wasn’t the only sense in which he was aggressive; Pearson fled to Las Vegas in 1962 after cracking the skull of a Nashville bookmaker with a golf club. And he moved to the desert city for good in 1963 after robbers ransacked his Nashville home.
Pearson was a rough character, but at the time poker was a rough business. Amarillo Slim was once robbed of Dollars 50,000 by three armed men who broke into the house where he was playing and seized the stakes on the table. In 1976, Slim tried to buy into a Las Vegas casino in partnership with another poker devotee, casino owner (and convicted murderer) Benny Binion, who hosted the first World Series tournament in 1970 at Binion’s Horseshoe casino. But Slim was convincingly put off after a visit from friends of Tony Spilotro, the most feared man in Vegas, who was reputed to have hung a 320lb man on a meat hook and tortured him.
The game didn’t begin to become safe for mathematicians until the late 1980s, when big entertainment corporations moved into Las Vegas and started to offer civilised ways to part people from their money. The geeks were welcome, but they were still struggling to turn game theory into wins at the poker table.
For many years after von Neumann’s death in 1957, academics were also struggling to apply game theory to real life problems of economics, biology and military strategy. One of the main difficulties was the sheer fallibility of human intellect. The joke about von Neumann being a demigod was spot on: he modelled his “zero sum” games as one demigod playing another, assuming that both players were as clever as he was himself. But excellent play cannot always assume an excellent opponent: there is no point in defending against brilliant strategies that your adversary is not smart enough to use.
This problem can be particularly acute in poker. A game- theoretically perfect poker strategy will pass up big opportunities against weak opponents who may bluff too much or too little. Punishing one mistake requires conservative play; punishing the other requires more aggression. Game theory assumes the mistakes will not be made.
Another difficulty for applying game theory that poker is so complex that even the fastest computer cannot find the optimal solution. Considering 10 possibilities a second, a player would have had to start calculating at the birth of the galaxy to find a game-theoretic solution for two players of Texas Hold ‘Em.
A real poker player who wanted to use von Neumann’s theories would somehow need to be able to perform calculations that were beyond even the great man himself. Or he would have to discover short cuts that simplified the mathematics without seriously damaging the quality of his play. And he would also need to recognise when the opponent was playing so badly that the game theory strategy needed to be set aside. It was going to take something special.
In 1988, the fledgling internet began to reverberate with a new phenomenon: IRC Poker. This was a simple program that used something called “internet relay chat”, a precursor of today’s online chat rooms, to deal cards and moderate a game of poker between internet players. There was no money at stake, just the chance to beat the world’s most obsessive, highly mathematical geeks.
Chris Ferguson soon emerged as a dominant player in this rarefied world. A computer-science graduate in the doctoral programme at University of California, Los Angeles, Ferguson was studying artificial intelligence, using game theory to help computers play board games. Ferguson was exposed to both poker and game theory at an early age. His family were avid games players, and his father was a maths professor who taught game theory at UCLA. On some weekends, the younger Ferguson drove to Las Vegas and covered his hotel bill by playing very conservative poker against the tourists. But IRC Poker was a much better laboratory for someone who wanted to get inside the game and see what made it tick. It produced raw data for him to analyse and enough fierce competition to keep him hooked.
He began using game theory to explore which hands to bluff with and how often to bluff, and the trade-offs between raising too little with a promising hand, versus raising too much and scaring people away. He memorised table after table of his results. His approach was pure von Neumann but, armed with powerful computers, he was able to analyse far more realistic poker games than his predecessor.
Soon he began to produce some unexpected conclusions. He showed that the old-school poker professionals were raising too much with strong hands. The idea was to win the pot while they were still ahead, but Ferguson showed that it was worth making smaller raises and encouraging opponents to stay in to try to improve their cards. Sometimes those opponents would get lucky, but on balance the strong hand would make more money with smaller raises.
“I showed a lot of my research to well-respected poker players,” Ferguson told me. “They pooh-poohed it, I think because they didn’t understand it and disagreed with the results. But I knew that what I was doing was accurate, and that disagreement showed that mathematics could outplay the best players in the world.”
That self-confidence is typical of Ferguson: he knew that game theory would give him an advantage, not because of his winnings at the table, but because the theory was right. By the time he outdrew T.J. Cloutier in 2000, few doubted that Ferguson had a formidable insight into the game.
By 2005, the World Series had outgrown the ageing Binion’s Horseshoe casino and moved to the Rio Hotel just off the Las Vegas Strip. At hundreds of tables, 6,000 contestants toyed with their clay chips, filling the huge hall with a sound like a swarm of demented crickets.
Many had qualified in online tournaments rather than paying the Dollars 10,000 entry fee – at peak times, more than 100,000 players around the world are playing internet poker for cash, a far cry from the old IRC days. Live televised poker has fed the demand, and been fed by it. After the networks started installing cameras to film each player’s hidden cards, the world’s worst spectator sport became a gripping contest, and players such as Ferguson became huge stars. When I walked the casino floor with him at the Rio he was approached every few seconds for pictures or autographs. He responded each time with practised good grace. “I enjoy it,” he confessed.
The new poker, open to anyone, has meant that cerebral players have had the courage to follow in Ferguson’s footsteps. “If poker were played only by outlaws in saloons, you wouldn’t get these doctors and lawyers here,” says Matt Matros, a mathematician from Yale and a poker writer who has won more than Dollars 750,000 playing in tournaments. Matros is certain that game theory is going to become a necessity to compete at the highest level of poker. So is Andy Bloch, an MIT graduate, game theorist and another tournament- winning poker player. Bloch believes that the two-person version of Texas Hold ‘Em will soon be solved, meaning that each decision at the table will have an answer that is known to be correct.
Game theory certainly confers the biggest advantage in the two- player game against the best players. Ferguson has reached both finals of a new head-to-head knockout all-star poker tournament, beating Cloutier on the way. If Bloch and Matros are right, every serious player will simply have to learn the moves for two-player poker, just as chess players memorise dozens of opening variations.
But most poker is not head-to-head, and most players are nowhere near von Neumann’s optimal play. Most of the new poker players are not expert mathematicians, but hopefuls with more money than sense. “If you want to play poker to make money, you’re doing it for the wrong reasons,” says Ferguson. “You have to love the game, and you have to like to work hard.”
Ferguson knows that amateur players spoil game theory’s assumption of expert play. Other top professionals believe that this will always limit the theory’s usefulness.
Howard Lederer, broad-shouldered and standing well over six feet, is nicknamed “The Professor” for his studied game and demeanour. When I buttonholed him at the Rio, he told me there were too many bad players around for game theory to be the main asset of a professional. “Pure game theory only comes into play against another great pro in a very pure situation. Basically, it’s the psychology of the game. You need to have a feel for the game theory, but psychology trumps game theory: dominating people at their moments of weakness in the tournament, getting to them.”
If Lederer is correct, the flood of new players is undermining the usefulness of game theory. Game theory tells you how to avoid losing to perfect play, not how to beat the weak players – known as the fish. The more fish enter the game, the less relevant game theory becomes. The poker legacy of von Neumann may therefore rest with an unusual new breed of players.
Back at Binion’s, just a short cab ride from the Rio, an alternative championship was being staged during the 2005 World Series. The scene was more internet cafe than casino, with the green baize hidden under a tangle of cables, six computers facing each other, invisibly playing dozens of hands a minute, dealing chips and betting with cards that existed only in cyberspace.
When Binion’s hosted the World Series of Poker in 1970, participation was by invitation only; a few hands were played and then everyone voted to honour the veteran Johnny Moss with the title of world champion. The 2005 World Poker Robot championship, the first such event, harked back to that tradition. The six software programs were there by invitation, and the true champion was not in doubt: the University of Alberta games research group, having defeated all electronic challengers for seven years, was asked to referee rather than play.
Darse Billings, the strongest poker player in the Alberta team and another game theorist, sat down with me at a nearby table, delighted to be speaking to someone who had heard of John von Neumann. With a smiling moon face covered by a short fuzzy beard, Billings has something of the teddy bear about him, but he claimed to be less than cuddly when the chips are down. “I used to be the top of the IRC Poker tables,” he said, with obvious pride. What happened? “Chris Ferguson started playing and he overtook me.”
Billings and his colleagues have yet to produce software capable of beating Ferguson, who is seen as a particular challenge because he is unfazed by an opponent who gives away no physical clues. But they relish the challenge of besting a world champion who holds a doctorate in artificial intelligence and game theory. For now, though, just about any top human player can outplay the robots. In a pair of exhibition matches concluding the World Poker Robot championships, the big-name professional Phil “Unabomber” Laak was recruited to play the machines. As a partisan crowd chanted “Hu- mans! Hu-mans!” he swiftly disposed of both the Alberta program and the newly minted world champion, a program called PokerPro. Nobody was surprised.
Artificial intelligence researchers see the same challenge in poker that von Neumann did nearly 80 years before them, that of understanding deception. At the moment von Neumann’s game theory remains the most successful approach, exemplified by the fearsome computer program, SparBot, which beats most of the humans who log on to the Alberta website to try their skill. “I believe that bots will eventually play better than all human beings,” predicts Billings. Ferguson agrees. “If poker robots had a tenth of the resources that were spent on chess, they’d already have beaten us.”
Many commentators now fear that the robots will destroy the online game that so enthused their creators in the days of IRC poker. Online poker players are thought to wager more than Dollars 250m a day – a tempting incentive to write a software program that could be let loose on unsuspecting “fish” all over the world. A decent poker player can make thousands of dollars a month playing the online game, so what if that player was replaced by an unlimited number of copies of a fiendish computer program?
Billings is convinced that the risk of this happening soon has been exaggerated. His own SparBot, an academic project, does not play for money, while he dismisses the other programs as simply not good enough. “The fear of bots is a much bigger problem than the threat of bots. There are dozens of poker robots out there on the internet, but all they are doing is contributing money to everyone else.” But he admits that it is only a matter of time before anyone will be able to download a free poker robot that will outplay the world champion. At that point, people may not care to risk money online against unidentified opponents.
One ironic possibility looms large: eventually, online poker will be dominated by the only poker players able to master John von Neumann’s game theory, the computers. Meanwhile, the humans will retreat back to the flesh-and-blood world of the casinos, where a nervous tic can tell more than a thousand calculations. Human computers such as Ferguson have carved out a niche, but as long as there are fish to hook at the table, the time-honoured skills of Puggy Pearson and Amarillo Slim will not be lost. Rather than conquering traditional poker, the biggest legacy of the poker mathematicians may be to make the game more exciting for the rest of us than ever before.
The most popular version of poker is Texas Hold ‘Em. Each player is dealt just two private cards face-down, instead of the five dealt in traditional poker. The other three are chosen from five shared or “communal” cards that are dealt face-up in stages to allow extra rounds of betting.
Players have four betting choices. They can “fold” (throw in the hand), “call” (match the existing largest bet), “raise” (increase the amount wagered) or “check” (opt not to bet – but only if there are no preceding bets in the round).
If one player bets and the rest fold, the better wins the round. If two or more players remain after the last round of betting, there is a “showdown” in which each player makes the best five-card hand from their two private cards and the communal shared cards on the table. The winner is the player with the best hand.
1. Royal flush
The highest ranking poker hand is five cards of the same suit – any suit – from 10 through Ace
2. Straight flush The next best hand is five cards of the same suit in sequence
3. Four of a kind Four cards of the same number or face, and one unmatched card
4. Full house Three cards of the same number or face, and two others of a different rank
5. Flush
Five cards of the same suit, not in sequence
6. Straight
Five cards ordered sequentially.
7. Three of a kind Three cards of the same face of number, plus two unmatched cards
8. Two pair Two pairs of cards of the same number or face, plus one unmatched card
9. Pair Two cards of the same number or face, plus three unmatched cards
10. High card
No matching cards. The lowest hand in poker. If two players have the same hand, the one with the highest card wins

Leave a Reply

Your email address will not be published. Required fields are marked *