Learning Center
Plans & pricing Sign in
Sign Out

Game Theory - AI



Dr Alexiei Dingli
 AI in Games before the 80s
• 1960's
   – First computer games
   – SpaceWar!
      • for two human players (1962)
   – Board games (e.g. chess) against the machine

• 1970's
   – Pong (1972)
      • Computer controlled opponent in arcade games
   – Space Invaders (1978)
      • Predefined patterns, no awareness
      • AI takes 1-2% of CPU
       AI in Games 80s
• Pac-Man (1980)
  – aware opponents with personality
• A computer beats a master chess
  player (1983)
• First fighting games
• Adventure games
  – Dungeon, Zork, ...
• First Multiplayer Online Role
  Playing Game
       AI in Games 90s
• First Person Shooter and Real Time
  Strategy games
• Games about/with evolution and
  – Creatures, Black&White
• Deep Blue beats Kasparov (1997)
• Graphic cards take the load off the
• AI takes 10-35% of CPU
Dune II
           AI in Games 2000
•   Computer games is a big industry

•   Games sell for about 25 billion USD per year

•   Market grows with 16% per year

•   A game project: 2 years, 8-15 million USD

•   Less cheating in AI

•   Characters are more aware

•   Characters collaborate better

•   Focus shift from graphics towards AI

•   Large part of the code is AI code (often made from scratch for
    each game)
Why is AI used in Games?
• Strategical/tactical decisions
  – Against or with you
  – Search for best counter action
  – Adaptivity

• Simulation
  – of natural behaviour
  – for animation (e.g. bird flocks)

• Shortest path problems
         Why is AI hard?
• Huge state space

• Huge action space

• Multiple tasks

• Non-deterministic
   – makes planning difficult
   – post-conditions difficult to set

• Often real time
Red Dead Redemption
Red Dead Redemption Map
The heart of the problem

   In a 1-agent setting, agent’s
     expected utility maximizing
       strategy is well-defined

  But in a multiagent system, the
      outcome may depend on
        others’ strategies also
•   Agent
     – player
•   Action
     – choice that agent can make at a point in the game
•   Strategy
     – mapping from history (to the extent that the agent can
       distinguish) to actions
•   Strategy set
     – strategies available to the agent
•   Strategy profile
     – one strategy for each agent

•   Agent’s utility is determined after each agent
    (including nature that is used to model
    uncertainty) has chosen its strategy, and game
    has been played
                     Game representations
                                                         Matrix form
             Extensive form                              (aka normal form
                                                         aka strategic form)

                                                                        player 2’s strategy
                             Left    1, 2
                                                             Left,      Left, Right, Right,
                  player 2                                   Left       Right Left Right
                             Right   3, 4              Up        1, 2    1, 2     3, 4    3, 4
                                            player 1’s
player 1                                    strategy
                             Left    5, 6
                                                      Down       5, 6    7, 8     5, 6    7, 8

           Down   player 2

                             Right   7, 8

                             Potential combinatorial explosion
Dominant strategy equilibrium
• Best response
   – si*: for all si’, ui(si*,s-i) ≥ ui(si’,s-i)

• Dominant strategy
   – si*: si* is a best response for all s-i
       • Does not always exist
       • Inferior strategies are called “dominated”

• Dominant strategy equilibrium is a strategy
  profile where each agent has picked its
  dominant strategy
   – Does not always exist
   – Requires no counter speculation
Prisoner’s Dilemma
    Prisoner’s Dilemma
• 2 prisons interrogated in separate

• Choices
  – Cooperate
     • Saying they’re innocent
  – Defecting
     • Implicating the partner in crime
    Prisoner’s Dilemma
• Base prison sentence is 5 years

• Payoffs they get
  – If they cooperate together (stay
     • Case is weakened
     • They only get 3 years each
     • Payoff of 2 years each
    Prisoner’s Dilemma
• Base prison sentence is 5 years

• Payoffs they get
  – If they both defect (accuse each
     • They get 4 years each
     • Payoff of 1 year each
    Prisoner’s Dilemma
• Base prison sentence is 5 years

• Payoffs they get
  – If one defects (one accuses the
     • He gets immunity
        – Payoff of 5 years
     • Other gets the full sentence of 5 years
   Prisoner’s Dilemma

How will you react if you’re one of
           the prisoners?
   Prisoner’s Dilemma

How will you react if you’re one of
           the prisoners?

  Statistics on a real experiment
   showed that 40% of prisoners
  involved in the game stay silent
   Nash equilibrium [Nash50]
• Sometimes an agent’s best response depends on
  others’ strategies: a dominant strategy does not exist

• A strategy profile is a Nash equilibrium if no player has
  incentive to deviate from his strategy given that others
  do not deviate: for every agent i, ui(si*,s-i) ≥ ui(si’,s-i) for
  all si’
   – Dominant strategy equilibria are Nash equilibria
       but not vice versa
   – Battle of the Sexes game
         • Has no dominant strategy equilibria

                          boxing                 ballet
       boxing               2, 1                  0, 0
        ballet               0, 0                 1, 2
     Criticisms of Nash
• Not unique in all games, e.g. Battle of the
   – Approaches for addressing this problem
      • Refinements of the equilibrium concept
          – Choose the Nash equilibrium with
            highest welfare
          – Subgame perfection
      • Focal points
      • Mediation
      • Communication
      • Convention
      • Learning

• Does not exist in all games

• May be hard to compute
   Existence of (pure strategy)
        Nash equilibrium

• IF a game is finite
  – and at every point in the game, the
    agent whose turn it is to move knows
    what moves have been played so far

• THEN the game has a (pure
  strategy) Nash equilibrium

• (solvable by minimax search at
  least as long as ties are ruled out)
       Some algorithms ...
• Finite State Machines
  – To handle behaviour

• A*
  – For short part problems

• Particle Methods
  – Flocking, etc
Finite-State Machines
  The Pac Man example
Example XXX
Example XXX
    Alpha-Beta pruning
• Problem with Minimax
  – Exponential number of moves

• Alpha-Beta pruning
  – Computes minimax without
    covering every node in the tree
  – Prunes away branches which will
    not influence the final decision
     Alpha-Beta pruning
• Alpha Cut-Off
   – we stop search of a particular branch
     because we see that we already have a
     better opportunity elsewhere

• Beta Cut-Off
   – we stop search of a particular branch
     because we see that the opponent
     already has a better opportunity

• Applying both is called Alpha-Beta
     Example Alpha-Beta

• Yield 3 irrespective of variables
   – max(    3,     min( -2       ,   x))
   – min(    3,     max( 8        ,   y))
Best path is not always the
      shortest path!
           Board games
•   Discrete time / turn based
•   Often deterministic
•   AI is in the opponent
•   Usually strives for optimality
•   Tree search
•   Library
         Role Playing and
        Adventure Games
• AI in enemies, bosses, party members
• Scripting, Finite State Machines,
• Role Playing ≠ Combat
   – combat oriented games are simpler to
• Conversations (grammar machines)
• Quest generators
• Towns
       Town Behaviour
• Need-based system
  – Needs (e.g. hunger, business, ...)
  – Actions (e.g. eating, trading, ...)
  – "Need pathfinding"
• Problems
  – Finding people
  – Unwanted interaction between Non-
    Player Characters
Strategy Games
• AI heavy (on both sides)
• Shortest path problems
• Strategical decisions
• Tactical decisions
• Town building and
  resource management
    – planning
• Indigenous life
• Reconnaissance (fog-of-
• Diplomacy
• Know thy enemy (observe
  and adapt)
First or Third Person Shooter
•   Enemies
•   Cooperative agents
•   Weapons
•   Attention
    – requires perception
       • requires a good physics engine
• Pathfinding
• Spatial reasoning
• Anticipation
           Racing Games
• Track AI
• Traffic (including pedestrians)
• Physics
• Tuning Non Player Characters and vehicle
• Genetic algorithms
• Particle swarm optimization
• Making realistic games requires more than good
• Computer controlled characters must behave
   – Naturally
   – Reasonably intelligent, without cheating!

• Graphics has dedicated hardware
   – More processing power avilable to AI

• In the future
   – Dedicated AI cards?
   – Combined AI/Physics/Graphics cards?
   – Multicore processors
Homework ...
Questions ?

To top