Soar Agent

Document Sample
Soar Agent Powered By Docstoc
        We are interfacing Soar to the BogWars game to program an agent to play the
game. The idea of the game is to collect the most gold possible. We played the game
using the human controls to try and discern the best strategies to program in our agent.
Based on the strategies we developed, we determined what inputs we should have to the
Soar agent, and what outputs the Soar agent should have. Certain things are much easier
to code in C++, so we plan on implementing them as derived inputs to the Soar agent and
actions that the Soar agent can choose.

Soar Agent
        The high-level planning and decision-making will be implemented in Soar. This
is where the strategies will be implemented. We view the role of the Soar agent as a
director, and it will put things on the output link that instruct the C++ code to do
complicated actions. For example, if the Soar agent decides that it wants to head towards
a certain item, then the C++ code will calculate a path and then follow it as long as the
Soar agent still wants to. The game state will also be pre-processed with the C++ code,
so it puts additional information on the input link, such as which monsters it will
encounter on its current path.

        We plan on having 3 top-level states: get gold, bring gold back, and level up. The
Soar agent will decide which goal it is trying to achieve based on a broad analysis of the
input, and this will be reevaluated at every turn. Note, however, that the name of the
state we are in does not necessarily indicate the action we take, since general rules can
override the state’s goal.

General Rules
          There are a few common sense rules that should fire no matter what state we are
in. First of all, if the boggle has low stamina, a reasonable amount of health, and cannot
make it to the home square, then wait. This is because you need stamina to both walk
and fight. It is silly to let your stamina run out because you will have to wait anyway. If
the boggle has the same conditions, but you can make it to the home square, then it
should head there. This is because stamina is charged faster on the home square than on
a regular tile. Also, the boggle should get power-ups if they are close and it needs them.
It is silly to get them when we do not need them because they will not be available to get
later. It is also silly to go far out of the way to get them because this will definitely
decrease stamina (which you might need) and will run the risk of engaging a monster,
which will decrease health. So we are planning on implementing the following rules to
get that behavior:
          If the boggle has low health, and there is a heart close to you, then get it.
          If the boggle has low stamina, and there is an apple close to you, then get it.

Get Gold
        This is the most important strategy in the game. Our rating is based solely on
how much gold we collect. We need to do this as fast as possible because the gold
decreases in value over time. First, we should define the concept of “guarded.” Gold is
considered guarded if there are monsters on (or near) the path from the current location to
the gold location. This should include monsters that move around, so we will use our
Monster Map (see C++ section) to determine this.
        To determine if we should take this action, we should go through each pile of gold
in order of distance (from closest to farthest) and issue a command to calculate a path to
that pile with A* and find the monsters that are guarding the path. If we find a pile that is
either unguarded, or there are only monsters that are a certain amount of levels (1 seems
reasonable or 0 when we are at level 9) lower than us or less, then we should get that pile
of gold now. We can battle the monsters we encounter on our way to get the gold
because we have determined that we have a good chance of winning. So in this case, we
will choose to follow the path; otherwise, we will reject that path and request a path to
another goal. The goal and path we rejected will no longer appear on the input link. This
strategy may have to be modified to account for lots of enemies on the path, or the
number of health that the boggle currently has. We may want to use our abstract
behavior defined for avoiding monsters when we have low health, and go collect a
power-up instead. It will definitely require some tweaking.

Return Gold
        When gold is collected, it is very important to return it right away. The reason
why we do not want to try and collect multiple piles of gold at once is because if we die
when we are carrying that gold, it is lost forever. The gold does not depreciate in value
that rapidly, so we feel it is advantageous for our agent to only risk losing 1 pile of gold
at a time. This strategy will take longer, but we have the opportunity to collect more
goals, so it seems better. It seems most intelligent to avoid all monsters on our path back
to the home square if possible. Although we could gain experience, we do not want to
fight monsters that have a good chance of beating us, and monsters that we are most
likely to beat will be worth very little experience. So, in this state, we will run A* with
modified tile costs, so that there is an increased cost for tiles containing monsters,
weighted by their difficulty. We will follow the A* path until there is a monster in the
way. We will then use our monster avoidance code in C++ until the monster is no longer
close, and then continue on the path.

Leveling Up
        We need to level up the boggle when there are no gold piles that it can safely
collect with its current level of experience, i.e., after we have rejected all paths to nearby
gold piles. After we level up, we can go back to the “get gold” state. This operator
should be proposed when we cannot get gold and we are not carrying gold.
        When playing the game, we found the most effective strategy is to go to an area
(close to a home base) where there are lots of enemies that respawn that can be beaten
easily. We also realized that it is beneficial when leveling up to just let the boggle die
when he has low health and stamina. This is because you get teleported to the home
base, and your health and energy are recharged after a short wait. If you walked to the
home base, it would take more time, and you would have to charge almost as much health
and stamina. Trying to get the boggle to reproduce this behavior is not as
straightforward, so this implementation is not decided completely yet. The following are
some thoughts we have about it.
        The boggle should move to an area with lots of respawning enemies that can be
beaten easily and fight the closest enemy to it. It should only try to move there if it can
get to the area without encountering enemies that can defeat it. This can be checked with
the Monster Map. It may be worthwhile to try and group several spawning points
together in a region, and try and go to those areas rather than regions with only one
spawning point or none. It is also important that it prefers to attack enemies that it can
get a good amount of experience from beating. Finding a close region with multiple
enemies (preferably respawning enemies) that are one level less would be ideal.
        If we cannot determine which enemies respawn, the boggle should fight the
enemies that are closest to it that are not too difficult. Eventually, it will slay the enemies
that do not respawn that are close, and keep heading to an area where the enemies
respawn. This is not preferred though because it is quicker to stay in an area with
respawning enemies because you get in more fights. Also, you do not have to walk much
to fight the next enemy, which is good because walking decreases your stamina, which
determines the damage you deal.

AI Functionality Programmed in C++
        There are a number of functions which will be useful for the Soar portion of the
AI to make use of, particularly those that involve searches of the map. These are much
simpler to implement and more efficient to perform in C++. The outputs of these
algorithms serve to provide more useful information about the game environment to the
Soar-based AI. Some must be specifically requested by the Soar agent and some will be
run every turn. If algorithm runtime becomes an issue, we may put a setting on the
output-link of the Soar agent, which switches the execution of each every-turn algorithm
on or off depending on whether the Soar agent decides that the information is needed.

Monster Map
        We will maintain a map (Note: do we need a full map or just a set of a few tiles?),
separate from the map which the game maintains, that identifies squares where monsters
are located. This information will be generated on the fly over the whole map and
updated every turn as needed. For each tile on this map, there will be a list of monsters
(C_Monster pointers) which threaten that tile. More specifically, each monster pointer
will be placed on a tile if, depending on script ID (written beneath with explanations):
        ID 0 (No movement): The monster is on this tile
               This is the only tile where you can encounter the monster.
        ID 1-12 (Static Scripting): The monster’s path includes this tile
               These are the only tiles where you can encounter the monster.
        ID 13 (Pursue): The monster’s chase radius includes this tile
               These are the only tiles where the monsters will come after you.
        ID 14 (Evade): N/A (no pointers will be placed)
               The monster runs away from you, so you won’t have to avoid it.
       ID 15 (Random): N/A (no pointers will be placed)
              It is too hard to predict where this monster will be, so any avoidance will
              have to be done on the fly when you get close to it (see Monster

Sort objects by Distance
        We will maintain lists of monsters, gold, home squares, spawn points, and power-
ups sorted by Euclidean distance from the player, in separate lists. This will enable us to,
for example, put the locations of the 5 closest gold piles on the Soar agent’s input link.
The path-finding A* algorithm can be quite time-consuming, and holds up the boggle for
turns at a time. Pruning away our possible navigational goals this way will quite
effectively limit the number of times we execute path-finding. The Soar agent will
choose one of these as a goal and ask for a path (see section on Soar strategies).

Calculating a path to a given goal, and “guarding” monsters
        For any given goal (particularly a home square (return gold), gold pile (get gold),
or spawn point (level up)), the Soar agent will request specifically when it wants a path
generated to that goal (via A* path-finding). The generation of this path will be
controlled by the C++ code, largely using the A* code provided with the package. In
addition, we will generate a list of the monsters whose pointers are located on the
Monster Map tiles corresponding to Map tiles visited by the path generated by A*. These
monsters are said to be “guarding” that goal. All of this path information will be
calculated and placed on the input link when the Soar agent requests it, and the Soar
agent will then decide whether or not to take that path.

Maintain Table of Monster Info
         This would be a compilation of knowledge, pre-set by the AI designers, of
statistics about the monster types based on human-intelligent observations of agent
behavior. This data can be unique per monster type, monster script ID, or both (a 2D
array). It is uncertain what data, if any, will be here in the end, but some potentially
useful info is:
         1) A rating of difficulty
         2) A radius of avoidance
         3) Maximum health
         4) Maximum stamina

Monster Avoidance
        The Soar agent may decide that the most important thing to do is to run away
from a particular enemy. This is especially likely to make sense if we are returning gold
and encounter a randomly-moving monster. In this case, it is hard for Soar to calculate a
direction of motion or a goal for path-finding, so we will calculate and follow a path
away from the monster in C++.

Soar/BogWars Interface with SGIO
        The interface between Soar and BogWars will be handled with SGIO, similarly to
the Soar/Unreal interface as described in "Using SGIO With UnrealScript.doc" found at

       Initialize Soar. Check every frame for changes in environment, adding and
updating input-link WME's as necessary. Commit changes to Working Memory with
Commit() function. Call ThinkSGIO() to run Soar until output is generated. Pull goal off
output-link with GetGoal(), and goal details with GetValue().

Functions that do not manipulate I-link
      StartUpConnection(AgentName, ConnType, IPAndPort);
      LoadProductions(ConnId, Productions);

Functions called to manipulate I-link, O-link

         Update<entity><attribute>(ConnId, id, value);
         Remove<entity>(ConnId, id);
               For example:
               id = AddNewItem(ConnId);
               UpdateItemAmount(ConnId, id, 500);
               RemoveItem(ConnId, id);
         GetValue(ConnId, ParamValue);

Input-Link Specifications

         ^bank                (int)
         ^force-rest          (yes/no)
         ^heading             (float)
         ^health              (int)
         ^last-heading        (float)
         ^level               (int)
         ^max-damage          (int)
         ^max-health          (int)
         ^max-stamina         (int)
         ^max-strength        (int)
         ^moving              (yes/no)
         ^next-strength       (int)
       ^player-number          (int)
               ^x              (int)
               ^y              (int)
       ^rand                   (float)
       ^stamina                (int)
       ^stamina-gain-rate      (int)
       ^stamina-loss-rate      (int)
       ^strength               (int)
       ^to-next-level          (int)
       ^to-next-strength       (int)
       ^treasure               (int)
       ^velocity               (float)
       ^xp                     (int)

                 ^heading      (float)
                 ^health       (int)
                 ^is-spawning (yes/no)
                 ^level        (int)
                 ^max-health (int)
                 ^max-stamina (int)
                 ^moving       (yes/no)
                         ^x    (int)
                         ^y    (int)
                 ^range        (float)
                 ^respawns     (yes/no)
                 ^script-id    (int)
                         ^x    (int)
                         ^y    (int)
                 ^stamina      (int)
                 ^strength     (int)
                 ^type         (int)
                 ^unique-id    (int)
                 ^velocity     (float)
                 ^xp-award     (int)
                 ^type         {gold, health, stamina}
                 ^amount       (int)
                 ^range        (float)
                 ^heading      (float)
                 ^astar-cost   (int) or (float)
                 ^guarded      (yes/no)
                        ^x     (int)
                        ^y     (int)
        ^enemy-id     (int) //list of all enemies along path
        ^id           (int)
                      ^x     (int)
                      ^y     (int)
              ^type {gold, health, stamina, home, spawn-point}

Output-Link Specifications
Because the Boggle can only perform one action, moving, there is no need to put
different commands on the output-link. We know the Boggle is either going to move, or
not, so we put a structure on the o-link that will tell the Boggle where to go.

        ^id            (int)
                ^x     (int)
                ^y     (int)
        ^status        {nil, received, complete, canceled}
        ^type          {gold, health, stamina, home, spawn-point}