evolutionary_computation_and_game_development-formal - Sergemail

Document Sample
evolutionary_computation_and_game_development-formal - Sergemail Powered By Docstoc
					Evolutionary Computation and
     Game Development
        Part 1: Background

•   What is Evolutionary Computation (EC)?
•   EC in the Real World
•   EC and the Gaming Industry
•   What is EC? (Continued)
•   EC and Games
•   Some Ideas
•   Questions
What is Evolutionary Computation?

• Evolutionary Computation is the study of algorithms
  inspired by the natural process of evolution.
   o Genetic Algorithms (GA)
   o Evolution Strategies (ES)
   o Evolutionary Programming (EP)
   o Genetic Programming (GP)
   o Particle Swarm Optimization (PSO)
   o Ant Systems (AS)
   o Ant Colony Optimization (ACO)
EC in the Real World

• Scheduling
  o Scheduling of flights
  o Task scheduling
• Routing
  o FedEx
  o UPS
  o Disney World Tour Planning
• Design
  o Aircraft design
  o Factory layout
EC in the Gaming Industry

    • IEEE Transactions on Computational Intelligence and AI in Games
    • Started in 2005
•   NERO with rtNEAT
•   Using EC to create actual AI opponents
•   Dynamic Content Creation
•   Play Testing
            NERO and rtNEAT

• NERO – Neuro-Evolving Robotic Operatives
• NEAT – Neuro-Evolution of Augmenting
• The NERO game was a sandbox where players
  evolved AI bots
  – Supervised learning
• Evolved Neural Network controllers with a
  Genetic Algorithm
           EC for AI Opponents

• RTS games have recently become a test-bed for
  evolving AI opponents
• There is also interest in evolving support
  functions, units, and features in RTS
• Research and competition have also been held
  and are still being held for AI bots in FPS games
• However, most research still center around
  classic board games
         Dynamic Content Creation

• Key questions
  –   How can a game keep the player engaged?
  –   How can a game keep players interested?
  –   How can a game challenge a player?
  –   How can a game satisfy a player?
• Two possible approaches
  – Adaptive difficulty or gameplay
  – Adaptive game content
                 Play Testing

• Use EC to find potential cheats or exploits
• Use EC to test feature balancing
• Use EC to possibly find better reproduction steps
  for isolating bugs/crashes
• One research done using Fifa ’99
  – EA Burnaby
What is EC? (Continued)

• Evolutionary Computation is the study of algorithms
  inspired by the natural process of evolution.
• So, what does that mean?
• Here's the High Level Pseudo-code
      1.Randomly Generate a Population (Generation)
      2.Evaluate Initial Population (Evaluation)
      3.Select Parents (Selection)
      4.Perform Crossover (Crossover)
      5.Apply Mutation (Mutation)
      6.Evaluate Offspring (Evaluation)
      7.Replace Parents (Replacement)
      8.Goto 3.
Characteristics of an Evolutionary
Algorithm (EA)
• "Any-time" Algorithms
   o The longer you run them, the better the solution (ideally)
• Stochastic in Nature
   o There is alot of randomness involved
   o This is the primary property
• Meta-heuristics
   o "a heuristic method for solving a very general class
     of computational problems by combining user-given black-
     box procedures in a hopefully efficient way"
• Flexibility
   • Black Box operator that can be swapped out
Black Box Components

• Problem Encoding
  o How is the problem encoded/represented
  o Genotype and Phenotype
• Operators
  o Selection
  o Crossover
  o Mutation
  o Replacement
  o Evaluation
• Candidate Solution
Problem Encoding

• Genotype - the representation of the problem that is being
  manipulated by the EA
• Phenotype - the physical representation of the problem
  that the genotype maps to
• Encoding is a key component
   o is important no matter what EA you are using
   o improper encoding will cause bad results
   o too complicated an encoding runs the risk of bogging
     down the algorithm
• There are guidelines, known methods, and encoding
  requirements based on the EA you want to use
• However, there will always be some black magic involved
      Example: Knapsack Problem
Sword            10
Shield           20
Helmet           4
Chain Mail       14
Gauntlet         6
Running Shoes    2
Healing Potion   1
Magic Wand       3
Healing Potion   1
            Knapsack Problem

• Large number of items, limited space
• Simplified form of a inventory/resource
  management problem
  – Inventory management in RPG’s
  – Resource allocation in RTS’s
         Phenotype vs. Genotype
Sword            10               1
Shield           20               1
Helmet           4                0
Chain Mail       14               1
Gauntlet         6                0
Running Shoes    2                1
Healing Potion   1                1
Magic Wand       3                0
Healing Potion   1                1
          Some Known Encodings

•   Bit Arrays
•   Integer/Floating Point Arrays
•   A mix of the above two
•   Finite State Machines
    – First used in Evolutionary Programming
• Expression Tree
    – Mainly used in Genetic Programming
Evaluation Function

• Just as important if not more important than how the
  problem is encoded
• The quality of the evaluation function directly influences the
   o EA's have no eyes
   o The evaluation function is an EA's guide dog
• There are also known good methods of formulating an
  evaluation function
• To get optimal results, the evaluation function must be
  customized for the problem
• There is a whole subfield of EC that doesn't use an
  evaluation function
   o Interactive Evolutionary Computation
       Knapsack Problem (Cont.)

• How would we evaluate a candidate solution?
  – How do we judge which item combination is better
    than any other?
  – Do we just want to fill the sack as much as possible,
    or is there a reason to prefer certain items over
        Knapsack Problem (cont.)
Sword            10                100
Shield           20                70
Helmet           4                 10
Chain Mail       14                54
Gauntlet         6                 35
Running Shoes    2                 27
Healing Potion   1                 49
Magic Wand       3                  5
Healing Potion   1                 49
                                                              Running Shoes

                                                                              Healing Potion

                                                                                                            Healing Potion
                                                                                               Magic Wand

                                                                                                                             Total Space

                                                                                                                                            Total Value
                                      Chain Mail


     Space 10 20    Shield   4        14            6         2                1               3            1                61
      Value 100 70           10       54           35         27              49               5            49                             399
Candidate 1 1    0           0         0            1         1                1               0            1                20            260
Candidate 2 1    1           0         1            0         1                1               0            0                47            300
Candidate 3 0    0           1         0            1         1                0               1            1                16            126
Candidate 4 0    1           0         1            1         1                1               1            1                47            289
Candidate 5 1    1           1         1            0         0                0               0            1                49            283
Candidate 6 1    0           0         0            0         0                0               1            1                14            154

•   Natural Selection
•   How do we select parents?
•   Which parents should be paired up for crossover?
•   Common selection methods
    o Random Selection
    o Rank-Based Selection
    o Roulette Wheel Selection
    o Tournament Selection
    o No Selection
Random Selection

                        P6   P1

                   P5             P2

                        P4   P3
Roulette Wheel Selection

                                  P1, 10
                        P6, 18

                                            P2, 15

               P5, 12

                                           P3, 20

                         P4, 25
Tournament Selection


                             P1        P4    P3


       P6              P2         P3

• How do we create offspring(s) with the parents?
• Completely dependent on problem representation
  and how many parents are involved
• Arrays of discrete values
   o Single Point or Multi-point Crossover
   o Uniform Crossover
• Array of real values
   o Discrete Crossover
   o Interpolation
• Trees or Finite State Machines
   o Pruning and Grafting

• Also representation dependent
• Any form of random perturbation
  o random bit flips
  o random value replacements
  o random insertions or deletions

•   This is where Natural Selection kicks in a second time
•   How will the offsprings replace the parents?
•   Who will survive?
•   Two common approaches
     o Offsprings replace parents completely
     o Offsprings and parents compete for survival

•   Evolutionary Algorithms are meta-heuristics
•   Operators are black boxes
•   Doesn’t matter how they are implemented
•   They just have to perform some specific task
    – The algorithm doesn’t care how task is accomplished
Additional Notes

• Crossover and mutation are sometimes considered optional
   o Some have found better results with only one of the two
• The truth is that there is alot of voodoo magic involved
   o Experience will tell you how best to encode a problem
   o But each problem type tends to have its own trick
   o There are known best practices for certain problem types
        Traveling Salesman
        Knapsack Problem
   o More often than not, your problem will fall into something
     totally new
          Precautionary Warning

• EC is a tool
  – Used for exploratory purposes
  – Used when a pretty good solution is needed in a short
    period of time
  – Used when there is no known solution
     • especially when there is no clear definition of what a
       solution is
• It will never perform better than a specialized
EC in Games

• Why is EC rarely used in real games?
• Complexity (Space and Time)
   – Which portion of the game do we want to evolve?
   – How would we encode the problem and where would we store the
   – Where will we find the clock cycles needed for the evolution?
   – What is a good evaluation function?
   – How would the producers tune this and what are behaviors they
     should expect?
• Uniform Experience
   – How do we control the overall user experience?
   – Can we even guarantee a predictable user experience?
   – Will predictability sacrifice algorithm performance?
Interactive Evolutionary Computation

• Replaces the evaluation function with some
  form of human response
  – Great for when the fitness cannot to
    quantitatively evaluated and qualitative
    measures are fuzzy at best
• The response can be a quantitative value or
  just a qualitative classification
• NERO is an example of an interactive EC
Adaptive Gaming

• Gameplay adapting to user action
  o Dynamic scaling of difficulty (already been done)
• The game adapts to user actions
  – Dynamic content generation
  – “No one really knows what they really want”
  – “They know what they want when they see it”