Poki The Poker Agent

Document Sample
Poki The Poker Agent Powered By Docstoc
					Poki: The Poker Agent

      Greg Priebe
      Zak Knudson
• Texas Hold’em poker
• Architecture and Opponent Modeling of
• Improvements from past Poki
• Betting strategies w/ analysis of hand
  strengths, weighting, and probabilities.
Texas Hold’em
• Each player gets 2
  “hole” cards
• Community cards:
  – 3 “flop” cards
  – 1 “turn” card
  – 1 “river” card
• Rounds of betting
  after each set of
Poki Program Architecture
Opponent Modeling
• Weight table
• No modeling
• Generic opponent
• Specific opponent
The Old Way
• Specific opponent modeling
• Betting frequency table
  – Keeps track of actions taken in different
  – Predicts median hand strength
• Left out many relevant details
New and Improved
•   Specific opponent modeling
•   Detailed
•   Dynamic learning/adaptation
•   Prelim. Neural net study
Poki’s Neural Net
Some Results
• Training Data
• Results from 80-
  90 percent
• Confusion matrix
• Error prone?
In Field Predictions
           #holdem1: 24% increase
In Field Performance
Small Bets Won

• Pro: .05 - .10
• Old opponent
  modeling: .09
• New opponent
  modeling: .22
Betting Strategy
• Pre-flop and Post-flop strategies are
  significantly different.
- Pre: little information available to influence
  the betting decision.
- Post: uses opponent models, private hand,
  and game context to generate an action.
Pre-flop Betting Strategy
* - pocket pair hand (two cards of the same rank)
s - suited hand
o - offsuit hand
Post-flop: Basic Betting Strategy
• Compute Poki’s effective hand strength
• Using the game context, betting rules, and
  formulas to translate the EHS into a
  probability triple
  { Pr(fold), Pr(call), Pr(raise) }.
• Generate a random number and use it to
  choose an action from the probability
Hand Strength
• Probability that a given hand is better than
  that of an active opponent.
Hand Potential
• With 2 cards yet to be revealed, we want to
  know the impact.
• Positive potential (Ppot) – chance hand
  improves to win.
• Negative potential (Npot) – chance hand ends
  up losing.
• Calculated by enumerating over all possible
  hole cards, and over all possible board cards.
• This is an expensive process.
Hand potential example
Effective Hand Strength
• Combines hand strength and potential
  to give Poki’s own relative strength
  against an opponent.
• Pr(win) = pr(ahead) * pr(opp doesn’t improve)
           + Pr(behind) * pr(we improve)
          = HS * (1 – Npot) + (1 – HS) * Ppot
Weighting the Enumerations
• Probability of hands played to a particular
  point will vary.
• To account for this, Poki maintains a weight
  table for each opponent.
• Need opponent modeling
• Complex system
• Betting strategies and determining hand
• Reweight helps to determine opponents
  likely action

Shared By: