Docstoc

games

Document Sample
games Powered By Docstoc
					Notes 6: Game-Playing



    ICS 171 Fall 2006




                        ICS-171:Notes 6: 1
                              Overview
•   Computer programs which play 2-player games
     – game-playing as search
     – with the complication of an opponent

•   General principles of game-playing and search
     – evaluation functions
     – minimax principle
     – alpha-beta-pruning
     – heuristic techniques

•   Status of Game-Playing Systems
     – in chess, checkers, backgammon, Othello, etc, computers routinely
        defeat leading world players

•   Applications?
     – think of “nature” as an opponent
     – economics, war-gaming, medical drug treatment

                                                                ICS-171:Notes 6: 2
                     Solving 2-players Games

•   Two players, perfect information
•   Examples:
     – e.g., chess, checkers, tic-tac-toe
•   configuration of the board = unique arrangement of “pieces”
•   Statement of Game as a Search Problem
     – States = board configurations
     – Operators = legal moves
     – Initial State = current configuration
     – Goal = winning configuration
     – payoff function = gives numerical value of outcome of the game
•   A working example: Grundy's game
     – Given a set of coins, a player takes a set and divides it into two
        unequal sets. The player who plays last, looses.




                                                                   ICS-171:Notes 6: 3
Grundy’s game - special case of nim




                                      ICS-171:Notes 6: 4
                 Games vs. search problems

•   "Unpredictable" opponent  specifying a move for every possible
    opponent reply
•

•   Time limits  unlikely to find goal, must approximate
•




                                                            ICS-171:Notes 6: 5
ICS-171:Notes 6: 6
Game Trees




             ICS-171:Notes 6: 7
ICS-171:Notes 6: 8
       An optimal procedure: The Min-Max method


•   Designed to find the optimal strategy for Max and find best move:

     – 1. Generate the whole game tree to leaves
     – 2. Apply utility (payoff) function to leaves
     – 3. Back-up values from leaves toward the root:
         • a Max node computes the max of its child values
         • a Min node computes the Min of its child values
     – 4. When value reaches the root: choose max value and the
       corresponding move.




                                                                  ICS-171:Notes 6: 10
                            Properties of minimax

•   Complete? Yes (if tree is finite)
•
•   Optimal? Yes (against an optimal opponent)
•
•   Time complexity? O(bm)
•
•   Space complexity? O(bm) (depth-first exploration)
•

•   For chess, b ≈ 35, m ≈100 for "reasonable" games
     exact solution completely infeasible
•

     –   Chess:
           • b ~ 35 (average branching factor)
           • d ~ 100 (depth of game tree for typical game)
           • bd ~ 35100 ~10154 nodes!!
     –   Tic-Tac-Toe
           • ~5 legal moves, total of 9 moves
           • 59 = 1,953,125
           • 9! = 362,880 (Computer goes first)
           • 8! = 40,320 (Computer goes second)              ICS-171:Notes 6: 11
       An optimal procedure: The Min-Max method


•   Designed to find the optimal strategy for Max and find best move:

     – 1. Generate the whole game tree to leaves
     – 2. Apply utility (payoff) function to leaves
     – 3. Back-up values from leaves toward the root:
         • a Max node computes the max of its child values
         • a Min node computes the Min of its child values
     – 4. When value reaches the root: choose max value and the
       corresponding move.


• However: It is impossible to develop the whole search
  tree, instead develop part of the tree and evaluate
  promise of leaves using a static evaluation function.

                                                                  ICS-171:Notes 6: 12
           Static (Heuristic) Evaluation Functions

•   An Evaluation Function:
      – estimates how good the current board configuration is for a player.
      – Typically, one figures how good it is for the player, and how good it
         is for the opponent, and subtracts the opponents score from the
         players
      – Othello: Number of white pieces - Number of black pieces
      – Chess: Value of all white pieces - Value of all black pieces
•   Typical values from -infinity (loss) to +infinity (win) or [-1, +1].
•   If the board evaluation is X for a player, it’s -X for the opponent
•   Example:
      – Evaluating chess boards,
      – Checkers
      – Tic-tac-toe




                                                                     ICS-171:Notes 6: 13
ICS-171:Notes 6: 14
Deeper Game Trees




                    ICS-171:Notes 6: 15
                Applying MiniMax to tic-tac-toe

•   The static evaluation function heuristic




                                                  ICS-171:Notes 6: 16
Backup Values




                ICS-171:Notes 6: 17
ICS-171:Notes 6: 18
ICS-171:Notes 6: 19
                   Pruning with Alpha/Beta

•   In Min-Max there is a separation between node generation and
    evaluation.




                             Backup Values




                                                             ICS-171:Notes 6: 20
                      Alpha Beta Procedure

•   Idea:
     – Do Depth first search to generate partial game tree,
     – Give static evaluation function to leaves,
     – compute bound on internal nodes.
•   Alpha, Beta bounds:
     – Alpha value for Max node means that Max real value is at least
        alpha.
     – Beta for Min node means that Min can guarantee a value below
        Beta.
•   Computation:
     – Alpha of a Max node is the maximum value of its seen children.
     – Beta of a Min node is the lowest value seen of its child node .




                                                                  ICS-171:Notes 6: 21
                          When to Prune




•   Pruning

    – Below a Min node whose beta value is lower than or equal to the
      alpha value of its ancestors.

    – Below a Max node having an alpha value greater than or equal to
      the beta value of any of its Min nodes ancestors.




                                                                ICS-171:Notes 6: 22
α-β pruning example




                      ICS-171:Notes 6: 23
α-β pruning example




                      ICS-171:Notes 6: 24
α-β pruning example




                      ICS-171:Notes 6: 25
α-β pruning example




                      ICS-171:Notes 6: 26
α-β pruning example




                      ICS-171:Notes 6: 27
                                Properties of α-β

•   Pruning does not affect final result
•

•   Good move ordering improves effectiveness of pruning
•

•   With "perfect ordering," time complexity = O(bm/2)
      doubles depth of search

•   A simple example of the value of reasoning about which computations are relevant (a
    form of metareasoning)
•




                                                                              ICS-171:Notes 6: 28
             Effectiveness of Alpha-Beta Search

•   Worst-Case
     – branches are ordered so that no pruning takes place. In this case
       alpha-beta gives no improvement over exhaustive search

•   Best-Case
     – each player’s best move is the left-most alternative (i.e., evaluated
       first)
     – in practice, performance is closer to best rather than worst-case

•   In practice often get O(b(d/2)) rather than O(bd)
     – this is the same as having a branching factor of sqrt(b),
          • since (sqrt(b))d = b(d/2)
          • i.e., we have effectively gone from b to square root of b
     – e.g., in chess go from b ~ 35 to b ~ 6
          • this permits much deeper search in the same amount of time



                                                                     ICS-171:Notes 6: 29
                           Why is it called α-β?

•   α is the value of the best (i.e.,
    highest-value) choice found so far
    at any choice point along the path
    for max
•

•   If v is worse than α, max will
    avoid it
•
      prune that branch
     

•   Define β similarly for min
•




                                                   ICS-171:Notes 6: 30
The α-β algorithm




                    ICS-171:Notes 6: 31
                               Resource limits

Suppose we have 100 secs, explore 104 nodes/sec
    106 nodes per move



Standard approach:

•   cutoff test:
     e.g., depth limit (perhaps add quiescence search)

•   evaluation function
     = estimated desirability of position




                                                         ICS-171:Notes 6: 32
                              Evaluation functions

•   For chess, typically linear weighted sum of features
                           Eval(s) = w1 f1(s) + w2 f2(s) + … + wn fn(s)



•   e.g., w1 = 9 with
    f1(s) = (number of white queens) – (number of black queens), etc.




                                                                          ICS-171:Notes 6: 33
                            Cutting off search

MinimaxCutoff is identical to MinimaxValue except
    1.  Terminal? is replaced by Cutoff?
    2.  Utility is replaced by Eval
    3.

Does it work in practice?

        bm = 106, b=35  m=4


4-ply lookahead is a hopeless chess player!

    –      4-ply ≈ human novice
    –      8-ply ≈ typical PC, human master
    –      12-ply ≈ Deep Blue, Kasparov
    –




                                                    ICS-171:Notes 6: 34
                 Deterministic games in practice

•   Checkers: Chinook ended 40-year-reign of human world champion Marion
    Tinsley in 1994. Used a precomputed endgame database defining perfect
    play for all positions involving 8 or fewer pieces on the board, a total of 444
    billion positions.
                     •
                     •
•   Chess: Deep Blue defeated human world champion Garry Kasparov in a
    six-game match in 1997. Deep Blue searches 200 million positions per
    second, uses very sophisticated evaluation, and undisclosed methods for
    extending some lines of search up to 40 ply.
•

•   Othello: human champions refuse to compete against computers, who are
    too good.
•

•   Go: human champions refuse to compete against computers, who are too
    bad. In go, b > 300, so most programs use pattern knowledge bases to
    suggest plausible moves.
•




                                                                         ICS-171:Notes 6: 35
              Iterative (Progressive) Deepening

•   In real games, there is usually a time limit T on making a move

•   How do we take this into account?
     – using alpha-beta we cannot use “partial” results with any confidence
       unless the full breadth of the tree has been searched
     – So, we could be conservative and set a conservative depth-limit
       which guarantees that we will find a move in time < T
        • disadvantage is that we may finish early, could do more search

•   In practice, iterative deepening search (IDS) is used
     – IDS runs depth-first search with an increasing depth-limit
     – when the clock runs out we use the solution found at the previous
        depth limit




                                                                  ICS-171:Notes 6: 36
              Heuristics and Game Tree Search

•   The Horizon Effect
     – sometimes there’s a major “effect” (such as a piece being captured)
       which is just “below” the depth to which the tree has been expanded
     – the computer cannot see that this major event could happen
     – it has a “limited horizon”
     – there are heuristics to try to follow certain branches more deeply to
       detect to such important events
     – this helps to avoid catastrophic losses due to “short-sightedness”

•   Heuristics for Tree Exploration
     – it may be better to explore some branches more deeply in the
       allotted time
     – various heuristics exist to identify “promising” branches




                                                                   ICS-171:Notes 6: 37
                                Summary
•   Game playing is best modeled as a search problem

•   Game trees represent alternate computer/opponent moves

•   Evaluation functions estimate the quality of a given board configuration
    for the Max player.

•   Minimax is a procedure which chooses moves by assuming that the
    opponent will always choose the move which is best for them

•   Alpha-Beta is a procedure which can prune large parts of the search
    tree and allow search to go deeper

•   For many well-known games, computer algorithms based on heuristic
    search match or out-perform human world experts.

•   Reading:R&N Chapter 5.




                                                                    ICS-171:Notes 6: 39

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:10
posted:11/6/2011
language:English
pages:37