Finate State Machines in Games by pptfiles


Next class (Tuesday) , you should have formed groups and
  select a topic for presentation among:

(1)    Real-Time Strategy games (Luis Villegas, Dulmovits, Alexander J): I: 8.2, 8.3, II: 7.5, 7.6, III: 4.7, 6.7; IV: 4.1,
       History of RTS games? - informed
(2)    Squad tactics (John Formica, Matt Mitchell,          David Pennenga ): I: 5.3, 5.4, 5.5. 5.6; II: 3.3, - informed
(3)    Scripting languages () I: Section 10 (7 chapters)
(4)     First Person Shooters (Michael Caffrey, Shamus Field, Jon Hardy): I: 8.1, 8.4; II: 7.1, II: 5.2, 5.3, history of
       FPS? - informed
(5)    Racing games () I: 9.1, 9.2, 9.3, 9.4, II: 8.1, 8.2,
(6)     Role Playing Games (Josh Westbrook, Ethan Harman) I: 8.5, history of RPGs? IV: “Stop Getting Side-Tracked
       by Side-Quests”, I: “Level-Of-Detail AI for a Large Role-Playing Game”, I: “A Dynamic Reputation System Based
       on Event Knowledge” - informed
(7)     Story line, drama (): IV: “Implementing Story-Driven Games with the Aid of Dynamical Policy Models”, IV:
       “Dialogue Managers”,
(8)    Sport Games (Daniel Phillips, Dan Ruthrauff) I: 9.7, II: 8.4, 8.5, - informed
(9)    Individual NPC behavior (Daniel Phang, Sui Ying Teoh): I: 2.4, 8.6, 9.6, II: 3.1, 3.2; IV: 5.6,, - informed
(10)    Player Modeling () IV 7.3, IV: “Player Modeling for Interactive Storytelling: A Practical Approach”, III:
       “Preference-Based Player Modeling”, II: “Player Modeling for Adaptive Games”,, “Dynamic Game
       Balancing: An Evaluation of User Satisfaction
       Gustavo Andrade, Geber Ramalho, and Alex Sandro Gomes, Universidade Federal de Pernambuco; Vincent
       Corruble, Université Paris 6”
(11)    Map, World, Wall generation ()
            Most Popular Games Genres
• Adventure games:
   – Solving puzzles
   – Finding clues
      • E.g., through conversations with NPCs
          – Classic: The Longest Journey
• Role Playing Games (RPG)
   – Player assumes fictional roles through Avatars
   – Avatar acquires skills by performing tasks/actions
   – “Levels” indicate the progress of the avatar
      • Classic: Diablo 2:
            Most Popular Games Genres (II)
• Strategy games:
   – resource gathering, managing economy, technology research
   – Real-Time Strategy (RTS)
      • Simultaneous actions (classic: Command & Conquer)
   – Turn-based Games (TBS)
       • Take turns (classic: Civilization II)
• Sport Games
   – Simulation of actual sport games (classic: FIFA series)
• Games of chance
   – Outcome highly influenced by a stochastic environment
• First-person shooter
   – Controlling avatar in first-person camera view
    (classic: Duke Nukem)
         Most Popular Games Genres (III)
• And many sub-genres
   – Combine and/or enhance main genres
     • Massive Multiplayer Online RPG (MMORPG)
         – RPGs with multiple other players in pervasive worlds
     • Stealth FPS
         – FPS emphasizing stealth over shooting
     • RTS/RPG
         – Combine both aspects (Warcraft III)
     • RTS/FPS
         – Combine both aspects (classic: Battlezone)
Finite State Machines in Games

               Original slides by: Jarret Raim
       FSM’s in Theory
• Simple theoretical
  – Set of states (S)
  – Input signals (I)
  – Transitional function
• A way of denoting
  how an NPC can
  change its state over
       FSM’s in Practice
• Each state represents some desired behavior.
• The transition resides across all states.
  – Each state “knows” how to transition to other states.
• Accepting states are considered to be the end of
  execution for an FSM.
  – Example of an accepting state in a game?
• Input to the FSM continues as long it has not
  reached an accepting state.
       FSM’s in Games
• Character AI can be
  modeled as a                          Monster In Sight
  sequence of mental
  states.                        Treasure

• World events can                          No Monster
  force a change in
  state.                 Monster Dead         Fight               Cornered

• The mental model is
  easy to grasp, even
  for non-programmers
  – This is important!
           FSM Example

                                              • States
          ~E   E,~D                             – E: enemy in sight
                                 E              – S: hear a sound
           E                                    – D: dead
                            ~S                • Events
               E        S                       – E: see an enemy
                                                – S: hear a sound
                                 S              – D: die
    D          Spawn
                 D                            • Action performed
                                                – On each transition
                                                – On each update in
                                                  some states (e.g.
            But it gets even easier (event-

If player walks                       Put 3 orcs
here then                             here

                               Starting place
                               of player
Even at a Larger Scale

                If player cross here
                Then declare war
       Ok Let Us Construct One Finite
       State Machine
• Lets program High Priestess Mar'li
  – Here is she in action

• Step 1: list states and events
• Step 2: Construct the Finite State Machine
       FSM Implementation - Code
• Simplest method          void RunLogic( int *state ) {
                              switch( *state ) {

• After an action, the           case 0: //Wander
  state might change.               if( SeeEnemy() )
                                       *state = 1;
• Requires a recompile              if( Dead() )
                                       *state = 2;
  for changes                       break;
                                 case 1: //Attack
• No pluggable AI                   Attack();
                                    *state = 0;
• Not accessible to non-            if( Dead() )
                                       *state = 2;
  programmers                       break;

• No set structure               case 3: //Dead

• Can be a bottleneck.        }
      FSM Implementation - Macros

• Forces structure       bool MyStateMachine::States(
                            StateMachineEvent event,
                            int state );
• Shallow learning       {
  curve                     State(0)
• More readable                   Wander();
                                  if( SeeEnemy() ) SetState(1);
  – Removes clutter by            if( Dead() ) SetState(2);
    using macros.              OnUpdate
• Easily debugged                 SetState(0);
                                  if( Dead() ) SetState(2);
  – Allows focus on         State(2)
    important code.               RotSlowly();
      FSM Implementation – Scripts

• Developer creates scripting language to
  control AI.
• Script is translated to C++ or bytecode.
• Requires a vocabulary for interacting with
  the game engine.
• A ‘glue layer’ must connect scripting
  vocabulary to game engine internals.
• Allows pluggable AI modules, even after
  the game has been released.
         FSM Processing
• Interpreted
  – Simple and easy to debug.
  – Inefficient since FSM’s are always evaluated.
       • Interpreted languages were the “rage” in AI
• Event Driven Model
  – FSM registers which events it is interested in.
  – Requires complex Observer model in engine.
  – Hard to balance granularity of event model.
• Multithreaded
  –   Each FSM assigned its own thread.
  –   Requires thread-safe communication.
  –   Conceptually elegant.
  –   Difficult to debug..
       Game Engine Interfacing
• Simple hard coded approach
  – Allows arbitrary                        Engine        AI
  – Requires full recompile
• Function pointers
  – Pointers are stored in a
                                  Engine            DLL             AI
    singleton or global
  – Implementation in Dynamic-
    link library
                                                Byte Code
     • Allows for pluggable AI.
     • E.g., The Tielt project       S. Interface          Compiler
• Data driven
                                       Engine                  AI
  – An interface must provide glue
    from engine to script engine.
      Optimization – Time Management

• Helps manage time spent in processing
• Scheduled Processing
  – Assigns a priority that decides how often that
    particular FSM is evaluated.
  – Results in uneven (unpredictable) CPU usage
    by the AI subsystem.
    • Can be mitigated using a load balancing algorithm.
• Time Bounded
  – Places a hard time bound on CPU usage.
    • Project # 1 will let you experience this
  – More complex: interruptible FSM’s
      Optimization – Level of Detail

• It’s ok to cut corners if the user won’t
• Each level of detail will require
  programmer time.
• The selection of which level to execute
  can be difficult.
• Many decisions cannot be approximated.
       FSM Extensions
• Extending States
  – Adding onEnter() and onExit() states can help
    handle state changes gracefully.
     • Example of when to use it in games?
• Stack Based FSM’s
  – Allows an AI to switch states, then return to a
    previous state.
  – Gives the AI ‘memory’
  – More realistic behavior
• Hierarchical FSM’s
               FSM Example
                              ~S       Attack-P
                                                  • Original version
                Attack                 E,S,~D       doesn’t remember
          ~E    E,~D          S
                                                    what the previous
                    D        E D       ~E    E
                                                    state was.
                                                  • One solution is to
                E        S
                                  D                 add another state
                                                    to remember if
                Spawn              S                you heard a
                                                    sound before
                 FSM Example
                                                                                        Worst case:
                            S        Attack-ES                                    Each extra state variable
             Attack-E                E,-D,S,-L
                                -S                   L     -E,-D,S,L               can add 2n extra states
             E,-D,-S,-L                                                              n = number of existing
                                L                -L                        E
                                                                     -E                      states
                E E              -L
Wander-L                                                                   Retreat-ES      and remembering N
-E,-D,-S,L         E                                             L          E,-D,S,L          actions into the
                                                                           -S                    pass is not
       -L                                                                                         possible!
                                                                                           (finite automata vs.
                       -E                                                                        pushdown
   Wander                                        -E E                     E,-D,-S,L
  -E,-D,-S,-L                                                                                    automata)
                                      D D
                   D                             D         Chase                      Using a stack allows to
                                   Spawn                 -E,-D,S,-L                   “remember” without the
                                     D           S                                         extra states.
             Stack FSM – Thief 3

Stack allows AI to                   Leads to more
 move back and                     realistic behavior
  forth between                          without
      states.                       increasing FSM
      Hierarchical FSMs
• Expand a state into its own sub-FSM
• Some events move you around the same
  level in the hierarchy, some move you up
  a level
• When entering a state, have to choose a
  state for it’s child in the hierarchy
  – Set a default, and always go to that
  – Random choice
  – Depends on the nature of the behavior
         Hierarchical FSM Example

              Wander              ~E       Attack

                                       E            ~S
             Pick-up                                     Chase
                                              D                  Spawn
Turn Right
                                   • Note: This is not a
             Go-through              complete FSM
               Door                    – All links between top level
                                         states still exist
                                       – Need more states for
              Markov Models : An instance of Non-
              Deterministic FSMs
                                                     • Selects transition based
                                                       on probability
                    Attack                             distributions
                                                     • Have multiple transitions
                    Approach                           for the same
              .3                                       (state,input) pair
  Aim &
Slide Right
                   .3                                • Adds variety to actions
 & Shoot                 .4
                                                     • Label each with a
                                                       probability that it will be
     Aim &                                             taken
    Slide Left
     & Shoot
                        Aim &                        • Probabilities can be
                        Jump &
                         Shoot                         learned (Reinforcement
          More FSM Extensions
• Fuzzy State Machines
  – Degrees of truth allow
    multiple FSM’s to
    contribute to character
• Multiple FSM’s
  – High level FSM
    coordinates several smaller
• Polymorphic FSM’s
  – Allows common behavior
    to be shared.
  – Soldier -> German ->
    Machine Gunner
            Polymorphic FSM Example


      Rifleman         Machine Gunner                Officer

American   German    American       German    American    German

 British    Soviet    British        Soviet    British     Soviet
      Debugging FSM’s
• Offline Debugging
  – Logging
  – Verbosity Levels
• Online Debugging
  – Graphical
    representation is
    modified based on
    AI state
  – Command line to
    modify AI behavior
    on the fly.
        Case Study: Robocode
• First determine what
  states are needed
   – Attack, Evade, Search,
• Code up the FSM
  transition function.
   – Include an error state
     that is noticeable.
• Setup debugging.
   – Verbosity levels are a
           Case Study: Robocode


Implement and
test each state                 A test bot AI
  separately.                  might help test
                             single behaviors.
                              (see Target bot)

       Defense and Firing Power
• Enable your bot to      void doMovement()
  dodge incoming             if (getTime()%20 == 0)
  fire.                      {
                             direction *= -1;
• Every 20 ‘ticks’           setAhead(direction*300);
  reverse direction.         setTurnRightRadians(
                             target.bearing + (PI/2));
• Adds a circle strafe.   }

• Selects a bullet
  power based on our      void doFirePower()
  distance away from         firePower =
  the target              }
• Reducing the scanner      void doScanner() {
                            double radarOffset;
  arc allows you to fire    if(getTime() - target.ctime > 4)
  faster.                      radarOffset = 360;
• If a target hasn’t been      radarOffset =
                               getRadarHeadingRadians() -
  seen recently, spin.         absbearing(getX(),getY(),targ
• Scan where the target     if( radarOffset < 0 )
  is.                          radarOffset -= PI/8;
• ‘Wobble’ to make sure        radarOffset += PI/8;
  to find the target.
• AI Game Programming Wisdom
• University of Wisconsin presentation
• Robocode Website

To top