VIEWS: 7 PAGES: 21 POSTED ON: 5/6/2010
The Smokin’ Cracks Dai Deh Joe Harrington Jeff Loman Mike Whittington A Game Theoretic Look at Texas Hold ‘Em A. Introduction The Popularity of Poker At one time, poker was a game played in smoky, underground card rooms and casinos, possibly with an Oreo-cookie-eating Russian or two thrown in for good measure. Today, the broadcasting of the World Series of Poker on ESPN has led to an explosion of popularity for poker and has made poker professionals, like Doyle Brunson, Phil Hellmuth and Phil Ivey, and amateur-turned-champions, like Chris Moneymaker, Greg Raymer and Joe Hachem, alike into household names. The 2008 World Series of Poker Main Event was a Texas Hold „Em tournament that drew 6358 entrants and paid out $8.25 million to the champion. Online poker gaming communities have sprouted up by the dozens, and the seminal poker book (or the bible, as some would call it), Doyle Brunson‟s Super System, has since been followed by hundreds of poker strategy books. How the Game Works There are many versions of poker that we can analyze: 5-card stud, 7-card stud, Omaha hi lo, razz, pot-limit Texas Hold „Em, and of course, no-limit Texas Hold „Em. For the purposes of this paper, we will analyze simple versions of Texas Hold „Em, and will assume basic knowledge of the rules. Complete rules can be found at: http://boardgames.about.com/cs/poker/a/texas_rules.htm Quick tutorial: Players are dealt 2 hole cards, which they can combine with 3 out of 5 “community cards” (that everyone has access to) to make their best 5-card hand possible. Typically, the two players after the dealer “button” pay small and big blinds (“ante” bets designed to encourage play). Players can bet 4 times – before the flop, after the flop, after the turn and after the river. In limit hold „em, players can bet strictly the amount of the big blind before and after the flop; and exactly 2x the amount of the big blind on the turn and river. In no-limit hold „em, players must bet a minimum of the big blind at each juncture, but are free to move “all-in” at any time (bet all their chips). Our Approach to Analyzing the Game The game of Texas Hold „Em is incredibly nuanced, which makes the science of analyzing optimal moves extremely challenging, both from a game theoretic approach and other strategic approaches. The right move in a particular situation can be influenced by the number of players sitting at a table or involved in a hand, betting patterns, probabilities, pot & hand odds, your betting position, and player styles/reputations, just to name a few. In an effort to hold these other factors constant and to zone in on specific situations where game theory can shed some light on poker strategy, we will simplify the game somewhat and isolate a handful of game situations. There have been many in-depth books written about poker strategy. Our second caveat is that reading our analysis will probably not make you into the next Internet poker superstar or get you to the final table at the WSOP main event. But it can give you a different spin on certain situations that come up in Texas Hold „Em and perhaps improve your play at your home game. Our audience is not the seasoned poker professional, but the MBA student who likes to scurry over to Vegas or the bumbling Game Theory professor who is continually outplayed at his home game. General Strategy A good poker player must have a basic command of the following skills: evaluating starting hands, observing opponents and making mental notes, and calculating odds. We want to quickly highlight the importance of these elements, which are critical to understanding optimal poker strategy from a non-game-theoretic point of view and avoiding the common pitfalls. Evaluating starting hands – The fundamentals of poker start with selecting to play with the proper starting hands. Every combination of hole cards can be force-ranked against the others by estimated value or by its long-term win percentage. For the purpose of analyzing starting hands with game theory, we will later divide the range of possible starting hands into 2 buckets – good starting hands (G) and bad starting hands (B). There can be different cutoff points for different players depending on their playing style. Observing Opponents – It can be difficult to read your opponent‟s playing styles because oftentimes you will receive very little information with which to deduce, or you‟re getting sporadic information that you can‟t interpret. For example, an opponent‟s actions may be a result of good or bad cards, a signal about their playing style, or a result of poor play. However, over the course of a card game (or several games, if you play with regular opponents in a home game), you will receive multiple cues which may allow you to draw conclusions about an opponent‟s playing style. If you can draw such a conclusion, you can exploit someone‟s style by adopting a style that is the foil to that particular playing style. Calculating Odds – After deciding whether to play with your starting hand, you will often be “drawing” to a better hand. For example, if you have AQ of hearts and the flop shows 2 hearts, then you have a flush draw. Ideally, you will be drawing towards the “nuts” (an unbeatable 5-card hand), but sometimes your draw will be towards a hand that can beat your opponent. “Hand odds” are important because they tell you your chances of making your hand. In the aforementioned case, your hand odds of hitting flush on the river are approximately 19% (9 outs out of 46 remaining cards). However, perhaps even more important is the ability to count “pot odds.” Pot odds are the ratio of the money in the pot to the bet in front of you. For example, if the pot is $25 and your opponent has bet $5 (total $30), then your pot odds to call are $30:$5 or 6:1. In other words, you have to win this hand 1 out of 7 times to break even, which is about 14%. In the heart flush draw example above (where you will hit your flush 19% of the time), the right play is to fold because the hand odds are greater than the pot odds. Fundamentally, if your pot odds are greater than your hand odds, then you will make a profit in the long run. Since the nature of poker is that you‟re going to play multiple hands, your long-term strategy should be that percentages will prevail over the long haul. B. Using Game Theory in Texas Hold ‘Em As discussed, understanding the odds of Texas Hold Em is useful for playing the game effectively. But if all players played strictly by the odds, the game would then be left up to chance. A player with the best hand would continue to push the pot limits until the pot odds are no longer favorable to others, resulting in their eventual folding. Consequently, the person who happens to draw the highest number of best hands over the course of the game would eventually win. But we know that this is not always true. In reality, every hand played affects every other hand. And since players rarely ever play strictly by the odds, the application of game theory can become very advantageous. Game theory can be applied to Texas Hold „Em in many ways across various situations. In this section, we will focus on a few that should give you a base understanding of how game theory can be utilized. For the sake of simplicity, we will first examine pre-flop strategies of a two player $0.50/$1 No-Limit Texas Hold „Em game, where a good hand is any set of two cards that has more than a 50% chance of winning and a bad hand is any set of two cards that has less than a 50% chance of winning (See Exhibit A for sample hands). Against Unaware Opponents with Pronounced Playing Styles By looking forward and reasoning back, there are several opportunities to exploit the playing styles of unaware opponents using game theory. Let‟s look at a scenario where you are playing against an aggressive, unaware player. This player is aggressive because she rarely folds and has tendencies to call most raises in order to see more cards, and she is unaware because she continues to play aggressive regardless of how you play your two cards. In this situation, you can exploit her tendencies by playing aggressive when you have a good hand and by playing conservative when you have a bad hand. For example, if you are dealt an Ace-King, you would want to raise because you have a high probability of winning and you know that she will call the majority of the time. Your goal at this point is to have her put as much money in the pot as possible. If she is dealt a bad hand, this strategy will pay off and you would secure a payoff of $2.50 (given the parameters of this example game) (See Exhibit B). If instead she is dealt a good hand and assuming that you both have an equal chance of winning, your expected payoffs are net zero. In the next round, let‟s say you are dealt a bad hand. In this case, you should play conservatively by folding or checking because you know that she will eventually raise if she has a good hand. If you fold 50% of the time and check 50% of the time, your expected payoffs are - $1.82 (given the parameters of this example game) (See Exhibit C). If instead she is also dealt a bad hand, your expected payoffs are again net zero because it is assumed that both players have a 50% of winning. Given that the combination of good and bad hands is equally likely, you should come out on top after several rounds of playing: Cards Payoffs (You vs Rival) Good vs Good $0 Good vs Bad $2.50 Bad vs Good - $1.82 Bad vs Bad $0 Total +$0.68 The concept of looking forward and reasoning back to exploit an unaware, aggressive player can also be applied to exploit an unaware, conservative player. A conservative player is one who is risk averse and will rarely call a big raise with a bad hand. In this case, you should exploit her tendencies by doing the opposite that we have recommended for an aggressive player: play conservative when you have a good hand and play aggressive when you have a bad hand. In other words, you should check and use small bets/raises more often when you have a good hand (to keep them in the game), and bluff more often when you have a bad hand. This game theoretic strategy should help you in the long run with positive payoffs. Against Sophisticated Opponents with Mixed Strategies Today, many players have played enough Texas Hold „Em to understand that they will likely lose if they play with a pronounced playing style. As a result, these sophisticated players employ mixed strategies to avoid getting exploited. A mixed strategy is when a player raises with a good hand (true raise) a portion of the time and raises with a bad hand (bluff) the rest of the time. He wants to bluff enough so that you will consider calling him when he has a good hand, but rarely enough so that you do not always call his bluffs. The end objective is to make it difficult to know whether you should fold or call when you get a raise. How should you play against a sophisticated player that employs mixed strategies? It depends on what cards you are dealt. For the sake of simplicity, let‟s examine the payoffs and equilibrium points under the definitions of good (>65% probability of winning), average (35- 65% probability of winning), and bad (<35% probability of winning) starting hands. When you have a good starting hand, your payoffs to fold and call a $4 raise is shown in Exhibit D. In this situation, the dominant strategy is for you to always call. Intuitively, you would never fold a good hand in the pre-flop betting round and this payoff table supports that notion. This outcome changes when you have an average or bad set of starting cards. In these situations, there is a mathematical equilibrium point that leaves you indifferent for calling or folding. When you have an average starting hand, your payoffs to fold and call a $4 raise is shown in Exhibit E. This game has no dominant strategy for either team. Your rival wants you to call his true raise, while you want to fold. In contrast, your rival wants you to fold when he bluffs, while you want to call him on it. With these payoffs as input into the calculation of an equilibrium point, you should call if your rival bluffs more than 22% of the time (see Exhibit E) and fold if he bluffs less than that amount. When you have a bad starting hand, there is again no dominant strategy for either team (see Exhibit F). But in this situation, you should only call if your rival bluffs more than 67% of the time. More generally, you should call more often when you have good starting cards against an opponent with an aggressive playing style. Mixed strategies apply to each stage of the game The examples we have detailed above talk about a simple example with two players and the first two cards. We have purposely neglected to talk about the remaining rounds (the flop, turn and river, because the mixed strategy mindset applies to each stage of a game. Individually, players are informed about their own cards and the number of potential outs. They should be calculating the pot odds, and the chances their hand is greater than the pot odds. As the odds change, your individual payoffs change, but this involves a recalculation based on the new assumptions and determining a new equilibrium. But fundamentally, it is the same process throughout the game. C. Signaling in Texas Hold ‘Em Poker is a Bayesian game where a player only knows information about herself, and her knowledge about the characteristics of the other players is incomplete. In this game, Nature randomly assigns a type to a player; in the game of poker, we can assign types by strong hand or weak hand. In one sense, we can compare a game of poker to the beer and quiche game (see appendix Exhibit G for a refresher on the model). Think of player 1 as the incumbent and player 2 as the potential entrant. In the beer and quiche game, player 2 could only observe what player 1 had for breakfast, beer or quiche. If player 1 is strong, he/she would rather have beer. Likewise, if player 1 is weak, he/she would prefer to eat quiche. However, player 2 will enter if he thinks player 1 is weak, and since the only message he gets is what player 1 has for breakfast, player 1 would likely rather have beer, regardless if he/she is strong or weak, and avoid the fight. Thus, there is a pooling equilibrium. Would player 1 ever wish to eat quiche? If player 2 believed that player 1 would only drink beer if they were weak. However, the strong player is always incentivized to deviate and drink beer, so eating quiche is not a pooling equilibrium. The nuances of Texas Hold „Em poker go beyond the beer and quiche model. In beer and quiche, both strong and weak players want to communicate that they are strong to avoid a fight, in poker player 1 does not always want player 2 to think he or she is any particular type. There are instances where a player, whether weak or strong, would play the same strategy. Let‟s assume a player has strong hand or a weak hand, and eating beer = betting high and eating quiche = folding. As shown in the table below, a player with a strong hand could play truthfully and bet high to signal the strength of his hand and discourage his opponents from staying in the game. A player with a weak hand, however, may take the same action and bet high but for a difference purpose. She may want to use a screening strategy and bet high to pressure her opponent to fold or to discover her rival‟s playing style. Naturally, players attempt to mix up their play to avoid information cascades. Poker players who play the same pure strategy instead of a mixed strategy are susceptible to having their rival, based on previous observations of my playing strategy, make a different choice than her own private signal (e.g. she will fold when I slow play if I habitually slow play, when she may have stayed in due to calculation of hand vs. pot odds). It should be noted that the skill of sending various signals noted is typically larger within no limit hold‟em. Limit hold „em marginalizes the payoff differences by restricting the amount you can raise each hand. If a player with a strong hand wishes to discourage others from staying in the hand, it is much more difficult to do so in the limit version because everyone will likely stay in longer. This makes intuitive sense when considering the size of everyone‟s bet to the pot. In no limit, there is obviously greater chance of intimidating other players out of the game. D. Adding Wrinkles to the Base Game Three-Player Games Thus far we‟ve restricted our analysis to situations with two people. However, most poker games begin with more than two people, sometimes with as many as 8 or 9 at a table. To assess how strategy changes we reexamine strategy with the addition of one additional person. The additional player could be a strategic mixed, aggressive, conservative or unknown type. With more players, hand odds decrease for any given hand and pot odds may increase for any given hand, though the hand odds effect is dominant. The decreasing hand odds are due primarily to the change from a two draw situation to a three draw situation. In other words, if hands could be force ranked on a scale of 1 to 100, then with two players the expected draw would be a 33 and 67 draw, with a 67 being the average leading draw. Now, the draw would be 25, 50, and 75, so the leading draw on average has changed from 2/3 to ¾. We begin with the assumption that if an opponent‟s hand and playing style is known. In this case, a distinct best response exists via a pure strategy – because no beliefs about the hand need to be formed, the hand is considered out of play. A hand is considered in play when the actual cards of a player are unknown. For example, an aggressive player‟s hand is only unknown before bidding, whereas a mixed is unknown both before and after bidding. Secondly, all hands are either biddable or not. Given this distinction the following could occur for any given competitive three-player scenario: 0 biddable hands (big blind takes little) 1 biddable hand (the biddable hand wins) 2 biddable hands (same as 2 person game) 3 biddable hands – new game If all opponents are conservative then the best strategy is equivalent to that of a two- person game against a conservative player (only the odds have changed) and an exploitive strategy should be used. The same is true when facing two aggressive opponents. This holds because they will each give the same signals and respond predictably. If any player still in play is either unknown or a mixed strategist, then the best response is a mixed strategy. This is because a mixed strategy is a best response to a mixed strategy. As for unknown players, since their signal value cannot be deciphered there are strategically equivalent to playing against a mixed player. Additionally, a properly employed mixed strategy cannot be beat, only matched. The only other possible scenario is when one is facing one aggressive and one conservative player with both opponents hands in play. If they are not in play (i.e. if one is bidding last), then their bids can be observed and their hands would be known. The same holds true if one goes second as one bid has been observed and thus the hand is known. Then a best response can be formed against it and the known type of player one is facing. However, when bidding first, one faces an unknown hand that can be played either aggressively or conservatively, which is the same situation one would face if facing a mixed opponent. Thus, a mixed strategy is the best response to this situation. Our analysis assumes perfectly signaling aggressive and conservative players. In reality they will only lean one direction or the other, which only bolsters the case for using a mixed strategy. In fact, the only time a pure strategy works is when facing uniform opponents utilizing known dominated strategies. Increasing Blinds We have assumed that blind levels are low and are insignificant in relation to stack size, however, blinds often increase as the game progresses and can become a significant portion of stack size, especially for languishing opponents. As blinds increase, signal strength can change, but it depends on the game in question. If the game is limit hold „em, signal strength remains because „all-in‟ was never an option. However, if the game is no limit, all in becomes informationally equivalent to a raise, as blinds become sufficiently high. This will serve to blur differences between styles. Conservative and aggressive players will be harder to distinguish, thus operationally, strategy against such opponents means mixed strategies will become more vital even if bidding in last position. The second effect is that as blinds reach a large enough size, the end of game becomes predictable. For instance, consider a $100 stack for each party with $50 blind. Either player can be blinded out in only two hands. Knowing this, each player will go „all in‟ on increasingly weak hands. These two forces severely punish losses in stack size to opponents. As this happens the game becomes „tippy‟. Wealth swings between players become more frequent and having a low stack requires an increasing number of consecutive wins just to reach parity. At the same time, blind size as percent of stack increases, forcing bids on weaker hands. Thus the game quickly reaches equilibrium where hands flop from either no bid or all-in bid on every hand until a winner is decided. Game theory can help us to look forward and reason back to determine our best strategy. Looking forward to the end before it happens and reasoning back forces us to conclude that taking risks on weaker hands earlier could yield advantage and playing more aggressive (i.e. more common all-in) yields greater advantage. If one is able to enter the two-person stage with a stack advantage, especially if coinciding with rising blinds, then significant advantage can be conferred. Not only this, but especially in a home game, the final two players may have discretion over blind increases and it would be to the advantage of the player with a bigger stack to accelerate the raises. Practical Considerations on Mixing This is not a purely randomizing function, for example, one should never fold with away pocket aces. The purpose of mixing is to expand the range of hands your opponents gauge you having for a given signal or the range of responses you will take to a given signal from them. Consider a game with 10 possible hands and 10 possible signals, each corresponding to the strength of a hand with a 10 being the best hand. A conservative player dealt a 5, would signal a 5. An aggressive player dealt a 5 might signal a 6. A mixed player, when dealt a 5, may choose to signal a 5 or 6. If a mixed player chooses to signal a 5, this leads opponents to estimate his hand as being either a 4 or a 5. Also, mixing should be individualized for each individual betting round. One option is to play one hand aggressive all the way through and another tight all the way through. However, if an aggressive bet is detected in the initial two cards, opponents knowing you will continue to be aggressive can exploit it. Consider this real poker example: one well-known poker strategy is to only play the top 15 hands and play them aggressively. One could do this, however anyone seeing this without a top 15 hand would fold. If you mixed in a few top 20 strategies, you would be able to „bluff‟ and get paid for worse chips. Note: Bluffing in this sense does not mean betting on a 2, 3. Exhibits Exhibit A. Sample Good and Bad Hands Good Hands Starting % chance Two Cards of winning AA 85 KK 82 QQ 80 JJ 78 1010 75 99 72 88 69 77 66 AK 65 AQ 64 AJ 64 A10 63 66 63 KQ 61 KJ 61 A9 61 K10 60 A8 60 55 60 Bad Hands Starting % chance Two Cards of winning 65 40 93 40 84 40 92 39 74 38 64 38 54 38 83 38 82 37 73 37 63 36 53 36 72 35 43 34 62 34 52 34 42 33 32 31 Exhibit B. Exploiting the Aggressive Player – Good Hand vs. Bad Hand Exhibit C. Exploiting the Aggressive Player – Bad Hand vs. Good Hand Exhibit D. Payoff Matrix if Raised and You have a Good Hand Exhibit E. Payoff Matrix if Raised and You have an Average Hand Your Best Response if You have an Average Hand Let p = % of the time that your rival bluffs Your expected payoffs are: p(-$4.00) + (1-p)(-$1.00) if you fold p($3.00) + (1-p)(-$3.00) if you call Therefore, you should call if: p(-$4.00) + (1-p)(-$1.00) < p($3.00) + (1-p)(-$3.00) -$3.00p - $1.00 < -$3.00 + $6.00p $9.00p > $2.00 p > 22%, meaning you should call if your rival bluffs more than 22% of the time Check: 22%(-$4.00) + (1-22%)(-$1.00) = -1.67 22%($3.00) + (1-22%)(-$3.00) = -1.67 Exhibit F. Payoff Matrix if Raised and You have a Bad Hand Your Best Response if You have a Bad Hand Let p = % of the time that your rival bluffs Your expected payoffs are: p(-$1.00) + (1-p)(-$1.00) if you fold p($0.00) + (1-p)(-$3.00) if you call Therefore, you should call if: p(-$1.00) + (1-p)(-$1.00) < p($0.00) + (1-p)(-$3.00) - $1.00 < -$3.00 + $3.00p $3.00p > $2.00 p > 67%, meaning you should call if your rival bluffs more than 67% of the time Check: 67%(-$1.00) + (1-67%)(-$1.00) = -1 67%($0.00) + (1-67%)(-$3.00) = -1 Exhibit G. Sample Beer and Quiche Model