# Games

Document Sample

```					 Games

Henry Kautz
ExpectiMiniMax: Alpha-Beta Pruning
•Cutoffs at Max and Min nodes work just as
before
•If range of values is bounded, can add
cutoffs to Chance nodes
•Assume that all branches not searched
have the worst-case result

•L = lowest value achievable (-10)
•U = highest value achievable (10)
ExpectiMiniMax: Cutoffs
• Alpha cutoff:

Values seen    Current    Values to come
value
• Beta cutoff:

Values seen    Current    Values to come
value
domain: Probabilistic STRIPS
Hungry Monkey         Planning
shake:   if (ontable)
Prob(2/3) -> +1 banana
Prob(1/3) -> no change
else
Prob(1/6) -> +1 banana
Prob(5/6) -> no change

jump: if (~ontable)
Prob(2/3) -> ontable
Prob(1/3) -> ~ontable
else
ontable
What is the expected reward?

[1] shake
[2] jump; shake
[3] jump; shake; shake;
[4] jump; if (~ontable){ jump; shake}
else { shake; shake }
ExpectiMax

ExpectiMax(n) 
U (n) if n is a terminal node
max{ExpectiMax( s) | s  children(n)} if n is max node
 P(s)ExpectiMax(s) if n is a chancenode
schildren( n )
Hungry Monkey: 2-Ply Game Tree

jump                          shake

1/3                  1/6            5/6
2/3

jump      shake jump            shake            shake jump             shake
jump

2/3 1/3 2/3 1/3 2/3 1/3 1/6 5/6 2/3 1/3 1/6 5/6 2/3 1/3 1/6 5/6

0   0    1    0   0   0     1     0   1    1     2   1   0    0     1     0
ExpectiMax 1 – Chance Nodes

jump                          shake

1/3                  1/6                       5/6
2/3

jump       shake jump                  shake            shake jump                        shake
jump

0         2/3           0         1/6        1         7/6             0             1/6
2/3 1/3 1/6 5/6                        2/3 1/3       1/6 5/6
2/3 1/3 2/3 1/3 2/3 1/3 1/6 5/6

0   0    1          0   0   0     1     0   1    1     2         1     0       0     1     0
ExpectiMax 2 – Max Nodes

jump                           shake

1/3                  1/6                           5/6
2/3

2/3                                          7/6                               1/6
1/6
jump           shake jump              shake                shake jump                        shake
jump

0         2/3           0          1/6       1             7/6             0             1/6
2/3 1/3 1/6 5/6                            2/3 1/3       1/6 5/6
2/3 1/3 2/3 1/3 2/3 1/3 1/6 5/6

0   0    1          0   0   0     1      0   1   1         2         1     0       0     1     0
ExpectiMax 3 – Chance Nodes

jump                           shake

1/2                                                1/3

1/3                  1/6                           5/6
2/3

2/3                                            7/6                               1/6
1/6
jump           shake jump                shake                shake jump                        shake
jump

0         2/3             0          1/6       1             7/6             0             1/6
2/3 1/3 1/6 5/6                              2/3 1/3       1/6 5/6
2/3 1/3 2/3 1/3 2/3 1/3 1/6 5/6

0   0    1          0    0    0     1      0   1   1         2         1     0       0     1     0
ExpectiMax 4 – Max Node
1/2
jump                             shake

1/2                                                  1/3

1/3                    1/6                           5/6
2/3

2/3                                              7/6                               1/6
1/6
jump           shake jump                shake                  shake jump                        shake
jump

0         2/3             0          1/6         1             7/6             0             1/6
2/3 1/3 1/6 5/6                                2/3 1/3       1/6 5/6
2/3 1/3 2/3 1/3 2/3 1/3 1/6 5/6

0   0    1          0    0    0     1      0   1     1         2         1     0       0     1     0
analysis is a conditional plan (also
Policies
called a policy):
Optimal plan for 2 steps: jump;
shake
Optimal plan for 3 steps:
jump; if (ontable) {shake;
shake}
else {jump; shake}
Probabilistic planning can be
generalized in many ways,
including action costs and hidden
state
• How much would you pay to play the
following game?
• Flip a coin. If heads, you win \$2.
• Otherwise: flip again. If heads, you win \$4.
• Otherwise: flip again. If heads, you win \$8.
• Otherwise: flip again. If heads, you win \$16.
Expected Value
• Expect value is INFINITE!
(1/2)*2 + (1/4)*4 + (1/8)*8 + …
• “Rationally” you should pay ANY fixed
amount.
• In real life, people will pay about \$20.
– This is consistent with logarithmic utility of
money
– (1/2)*log(2) + (1/4)*log(4) + (1/8)*log(8) + …

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 3 posted: 6/4/2012 language: pages: 58