Coordination by yaosaigeng


									             AI for Computer Games
   Basic ideas and examples
   Control and multi-agent systems
   Multi-tier coordination
    »   General ideas
    »   In the dungeon
    »   Scripted Coordination
    »   Group pathfinding
   Emergent coordination
   Coordination with human players
   Agent coordination in real world
            Teams, squads, factions
   In a game can have several or many NPCs
    coordinating action.

    » The NPCs may be coordinating with each other.

    » The NPCs may be coordinating with human players.

   May also have human players cooperating.

   There will often also be competition.

      Coord. example: football game
   Example. Football game:

    » One team formed entirely from NPCs
       – each NPC performing actions to win game for team.

    » Second team contains characters which are (partly)
      controlled by a human player.
       – It may be that the human does not control all team members at
         all times
       – Some lower-level (e.g. reflex) behaviours may be automatic.

    Coord. example: real-time strategy
   Dune II (1992). Direct ancestor of many moden RTS

    » Two factions at any stage. Each gather resources and try to
      destroy opposition faction.

    » A faction consists of a base, plus soldiers, vehicles etc.

    » Human player controls one faction, builds base, moves
      vehicles etc.

    » Computer controls other faction.

Dune II (Factions)

Dune II (Agents)

      Coord example: background characters

   Background NPCs/mobs should look realistic
    » Flocks of prey animals (e.g. sheep): stay close, no formation,
      slow movement.
    » Swarms (e.g. bees): stay close, no formation, fast indivdual
      change of direction.
    » Flocks of birds: close, almost in formation
    » Populations of background people (extras):
        – Lots of both simple and complicated social effects.
   In real-life each social individual does not act
    completely independently, but conditions its
    behaviour on what others do.
   Need to code this so that it looks appropriate for the
    given game.
               Cybernetics – Control
   A designer often wants to control a system to
    achieve a goal: e.g. reach or stay in a state, travel a
    certain trajectory.
    » Land on moon, keep reactor cool, intercept aircraft.
    » You’ve been coding with similar ideas already: pathfinding, pursuit.
   Sometimes what we have to control is a multi-agent
    » The system contains multiple agents, which have a degree of
      autonomy in their chosen behaviour.
    » We may be able to design them, or have some degree of control
      over them or their environment.
    » Ecology, economy, robot football team, automated software

             Centralization of Control
   Consider a system designer, who has to arrange
    (control) several agents to achieve some goal. There
    are two extreme approaches:
   Centralized: have a central planner that decides and
    dictates what the agents should do.
    » Economics: planned economy
    » Military command
   Decentralized: allow the agents to make their own
    »   Economics: free market
    »   Design individual agents (if artificial)
    »   Control aspects of agent environment (interest rates, rewards, etc.)
    »   Design communication mechanisms between agents.
   In reality, the chosen solution is often a complex mix.
   What works well depends a lot upon context.
        Game Coordination Methods
Again, there are two extremes.
 Multi-tier: a multi-level version of centralized control. High-level AI
   gives orders to AIs at lower level, which in turn give order to even
   lower-level AIs, until orders reache bottom-level NPCs that issue no
   orders, but just perform actions.
     » This is sometimes called top-down multi-tier.

   Emergent: a version of decentralized control.
     » Individual NPC’s adapt their actions to what other NPCs are doing,
       but there is no direct high-level control.

   There are also complexes mixes, for example Bottom-up multi-tier:
     » Individual NPCs take individual decisions, but (in some way) check
       back with their superior(s) that their decisions are acceptable.

   Agent wants to achieve some goal state.

   Has various actions that change the state.

   Tries to find a (good) sequence of actions to get to
    the goal.

                     Multi-tier AI
   High-level strategic AI makes plans.
   Part of plans passed as orders to AI at a lower level
    » Delegation
   Eventually reaches individual NPC level
   NPC acts

   Two-level case:
    » Planner is similar to human giving orders to NPCs
    » For example in RTS games.

Example: RTS Village

        * Screen from Age of Empires III
                       RTS Example
   Planner AI:
    » Looks at game state. Sees that certain resources (e.g. grain,
      wood) are in short supply.
    » Has some underlying economic model.
    » Formulates plan/makes decision about what to do.
    » Issues orders: sends three villagers to build farms and two to
      cut down trees
   Individual NPC (villager):
    » Executes `farmer/farm builder’ or `woodcutter’ behaviour,
      as directed.
    » Farmer: finds spot, creates farm
        – individual decision, e.g. based on terrain analysis
    » Woodcutter: goes to trees, cuts, carries, drops, continues.

          Multi-tier in the Dungeon
   Factions are the high-level AI
   Go and have a look at the Faction class.
   Factions have:
    » A name
        – This must be the name of a colour
    » A colour
        – This is induced from the name (you don’t need to specify it)
    » A behaviour
        – This comes from a behaviour class
        – Behaviour is of class Behaviour, exactly as for individuals.
        – Choice of behaviour class specified in XML file.
    » An onTick method
        – Called by game

                     Faction XML
   See the document FactionXML, accessible via the
    course Information page, for instructions on how to
    specify factions and faction-creatures.

   Basically:
    » Specify Factions with Name and Behaviour fields.
    » Specify Creatures with Faction and Behaviour fields.

                   Faction Behaviour
   Look at the Game class.

   tick method from Game object calls faction.onTick for every faction
    before if calls mob.onTick for all the mobs
     » Faction behaviours all run before behaviours of individual NPCs.

   Faction gives orders to its NPCs

     » Need an API for communication between faction and NPC
       behaviour classes
        – Simple example: faction to set goal for individual NPC.

          Dungeon Faction Example
   Assume tournament where goal is to pick up the
    most gold (no combat).

   Two Factions (Red, Blue)
    » Team of NPC’s on each side

   High-level (faction) strategy:
    » In which rooms should each of my NPCs work?

   Low-level (individual NPC) strategy:
    » How should I pick up gold in my assigned room?

Dungeon Example (continued)

              Techniques for Multi-tier
   High-level AI: the central planner is designed to achieve some complex
    strategic objective(s).
    –   Complex, model-based strategic analysis, decision-making, learning, and
        planning may be done.
    –   May also be some simple reflexes.

   Low-level NPC AI:
    –   Sometimes no real AI, just follow script
    –   simple decision-making (FSM, decision tree, etc)
    –   More recently: some complex decisions and planning

   Computational resource may constrain ability to have have large
    numbers of complex AI NPCs at low-level
»   Large games use a scheduler to handle events that change the game state
    (rather than polling every NPC and other game items at every tick as in the

               More on Scheduling
   Divide-up processor time between various
    computational tasks.
    » Computations by high-level AI
    » Computations for the various low-level AI NPCs
   Example

   Clever tricks possible based on maintaining game
    quality, exploiting player focus etc.
        Implementation of multi-tier
   Basically need two things (according to Millington).

   Communications:
    » orders and information need to be passed down hierarchy
        – Sometimes information (feedback) may need to be relayed
          back up.

   (Hierarchical) scheduling:
    » Bottom-level individual NPCs must display the correct
      behaviours, in the correct sequence as ordered, and in
      concert with others, as directed.

                Concurrency Issues
   Our NPCs are concurrently acting entities,
    communicating and coordinating.
    » Issues, tools, techniques from concurrent computation
      relevant to implementation of multi-tier.
   Synchronizations:
    » Soldiers who attack simultaneously
    » Infantry attack after cavalry, and cavalry after longbow.
   Resource transfers
    » Resource gathered by one agent (e.g. woodcutter) and
      passed to other (e.g. house builder).
   However, low-level agents may well be more flexible
    and used in less predictable environments than
    standard concurrent programs, so coordination is
    often more sophisticated and dynamic.
     Scripted Coordination Example
   Use real-life military manuals on squad formation,
    e.g. how individual soldiers should `clear’ a room.

       Designing scripted behaviour
   Sometimes used to prescribe actions of large

   The Art of War by SunTzu was used to give the
    scripted battlefield tactics in the game Shogun.

                 Group Pathfinding
   NPC’s in group: find path for group as whole.

   Solution are essentially the same as pathfinding for
    the individual, but with a few tweaks
» Avoid getting in each other’s way
» Coordinate so that travel efficiently through restricted spaces
        – E.g. tank column going through narrow pass
» Plot route that is suitable for all members of group
        – E.g. heavy weapons can’t cross rough ground that infantry can
» Travel in a sensible formation
        – E.g. defend against ambush

                Group Pathfinding
   Most games seem to use variants of A* for the group
    problem as well as the individual problem.

   Main (stupidity) problem is deadlock, particularly
    around bottlenecks.
    » Various context-sensitive hacks to deal with this.

   A* group pathfinding demo:

                 Group Pathfinding
   Most basic idea: plot path for each group member
    » Eliminate location nodes according to strongest constraints
      of all group members.
        – E.g. If widest can’t go down corridor, then (maybe) reject route
          for all.
    » A lot of calculation if many members.
    » Other members are obstacles.
    » These obstacles move, so may need to re-plan path.
   Efficiency saving: pathfind for one unit, then vary first
    few, and last few, steps for every other unit.
   Make more realistic by having a few variations along
    the path for a few units.
Group pathfinding: follow-the-leader
   Another simple idea is to nominate a leader for the
    group. Have it do the pathfinding, and then make
    others follow it.
    » The following may be done directly, having units follow
      behind in a queue.
    » May be done by flocking (see below).

   Similar idea can also be used for other behaviour.

            Emergent Coordination
   No overall controlling AI
   Individual NPC AIs take into account what their
    teammates are doing
    » Like players in football match

                Example: Flocking
   Group of NPC’s moving together
   Reynold’s algorithm
    » Each NPC maintains a minimum separation from the others
    » Each NPC tries to move with same velocity (direction and
      speed) as the flock
    » Each NPC steers towards the centre of the flock.

  Go and look at the boids animations at

         Dungeon Combat Example
   Dungeon with hero for faction to fight.

   Individual Orc strategy: 2 states
    » AVOID-COMBAT: move away from enemy
    » ATTACK: attack enemy

   Individual Orc uses rules:
    » AVOID-COMBAT: if alone
    » ATTACK: if at least one other team-mate present

   Lots of details and improvements possible depending
    on other game details.

         Dungeon Combat Example
   Result is coordinated behaviour
    » Orc avoids hero until teammate arrives on scene, then joint
      attack on hero

   Not explicitly directed by a higher-level AI
    » “emergent” group behaviour
    » common in real world: animals, humans, …

   Group behaviour is very different, depending upon
    » basic behaviours of each agent
    » inter-agent contingent behaviour.

    Coordination with Human Player
   Player can be top decision-maker in multi-tier

    » Real-time strategy

    » Works well

          Coordination with Player
   Harder if player is part of a team, not the top boss

    » Multi-tier doesn’t work well unless player robotically follows
      orders (no fun)

    » Emergent might work if NPCs in practice follow players lead
      (even if notionally he is not the boss)

   Major challenge!

            Combining approaches
   Emergent techniques often work best (in computer
    games) with a small number of characters.

   Divide faction into groups of NPCs

    » Multi-tier used within each group

    » Emergent behaviour between groups.

                   Example: Squads
   Strategy game, each side has 40 characters
    organised into 4 squads

    » Squad-level AI is emergent,
        – there is no higher level AI giving orders to squads

    » However, within any squad, the individual NPCs follow
      orders from the squad-level AI.

                Intelligent Agents
   NPCs in computer games are a type of intelligent
    software agent

    » Autonomous entity which observes and acts in an
      environment, and tries to achieve a goal

   Agents can interact with other agents
    » Compete, cooperate, coordinate…

           Ex: autonomous vehicle
   Goal: drive a robot car to complete a course quickly

   Environment: world, car

    » Observe: outside world, using various sensors (cameras,

    » Act: change car’s speed, direction

   Interaction: with other robot cars

    » don’t crash, stay in convoy, …

                   Ex: travel agent
   Goal: find cheapest ticket which matches user needs,
    given constraints.

   Environment: airline websites

    » Observe: get info on prices

    » Act: make actual booking

   Interaction: with agents from other people (get
    advice on best deal)

    » Negotiate with airline website agent?

                   Example: auction
   Goal: obtain resource cheaply, by bidding in an
    » Resource might be: processor power, bandwidth

   Environment: auction

    » Observe: what other participants do

    » Act: place bid

   Interaction:
    » Other participants, auctioneer

        Agent Interaction: approach
   Consider robot car

    » Emergent behaviour:

        – observe what other cars are doing

        – communication: turn indicators, horn, …

    » Multi-tier behaviour:

        – AI for convoy gives orders to individual cars

        – AI for logistics tells lorries where to go.

         Communication Protocols
   Turn signals
    » Work well for humans
    » Robots cannot see as well

   Better to post a message on Internet?
    » “I will turn left” message
    » Website? Network packet? Text message???
    » Must agree on a protocol

             Real-world Challenges
   Uncertain information
    » Its raining, my vision system can’t tell what the other car
      actually doing

   Incomplete knowledge
    » Roadworks which are not on my map

   Comms failures
    » I can’t send the other car a turn message, because no
      mobile/wifi/etc coverage

   Trust and security
    » Do other agents perform as expected?
    » Can anyone else interfere with comms?
              Real-world challenges
   Problems for agent interacting with humans

    » They don’t understand my communications
        – e.g., network packet warning of a turn

    » I don’t understand their communications
        – e.g., gestures they make

    » They act unpredictably
        – Random failures
        – They can be untrustworthy: cheat, get lazy, take short-cuts, etc.

        General Agent Coordination
   Possible techniques:
    » Coordination through partial global planning
        – Individuals form plans about how to solve bits of larger problem
    » Coordination through joint intentions
        – Individual intend to cooperate (it is a shared goal)
    » Coordination by mutual modelling
        – Individuals have understanding and models of others
    » Coordination by norms and social laws
        – Norms: social conventions that individuals usually stick-to
        – Social laws: just that
    » Multiagent planning and synchronization
        – Centralized planning for distributed plans (master/slaves)
        – Distributed planning (team of specialists planners, separate
          execution agents)
        – Distributed planning for distributed plans (team of
          planners/execution agents cooperating, negotiating etc.)
                   Agent Research
   Large research group at Aberdeen CS

   Some use of computer games
    » User multi-player shooter (OpenArena) to test out ideas

  Millington section 6.4 and 9.1.4
 Mike Wooldridge’s book: An Introduction to
   MultiAgent Systems, 2nd ed., Wiley, 2009,
 the accompanying lecture notes on agents:

   Critical Mass, Philip Ball. Popular account of
    emergent cooperative phenomena in physics, biology
    and society.


To top