Genetic Algorithms, Search Algorithms
Jae C. Oh
Fogel in 1960s
Individuals are encoded to be finite state
Rechenberg, Schwefel in 1960s
Real-valued parameter optmization
Holland in 1960s
Wide variety of evolutionary algorithms
No one seriously tries to distinguish them
except for some cases and purposes.
We will call all Evolutionary Algorithms
And I will call them Genetic Algorithms or
Evolutionary Algorithms for generic terms
Notion of Search Space
Real world problem
Abstraction -> State Space
Exploring the state space for given
problem Search Algorithms
Finding (through search) a suitable
program, algorithm, function for a given
Training Data Learning Algorithm Program
Learning Algorithms (function
Set of Hypothesis
Learning Algorithms (Digression)
How do we know the found hypothesis,
program, function, etc. are the one we
are looking for?
We don’t know for sure
Is there any mathematical way of telling
how good hypothesis is?
I.e., |h(x) – f(x)| = ?
Computational Learning Theory can tell us
What are Genetic Algorithms?
Find solutions for a problem with the idea of
evolution. Search and optimization techniques
based on Darwin’s Principle of Natural
Randomized search and optimization algorithms
guided by the principle of Darwin’s natural selection:
Survival of fittest.
Evolve potential solutions
Mutations? Randomized, parallel search
Models natural selection
Uses fitness to guide search
Evolution is a search process
From the Tree of the Life Website,
University of Arizona
Orangutan Gorilla Chimpanzee Human
Evolution is parallel search
AAGGCCT TGGACTT AGTGACCA TGGACTA
AGGGCAT TAGCCCT AGCACTT AGGGCAA CAGCACCA AGCACTA
AGGGCAT TAGCCCA TAGACTT AGCACAA AGCGCTT TCGCCCA AGTACAA
AAGGCAA TAGGCCTA AGTGCTA
Genetic Algorithm Overview
1. Starting with a subset of n randomly chosen
solutions ( )from the search space (i.e.
chromosomes). This is the population
2. This population is used to produce a next
generation of individuals by reproduction
3. Individuals with a higher fitness (| - |)have
more chance to reproduce (i.e. natural selection)
GA in Pseudo code
0 START : Create random population of n chromosomes
1 FITNESS : Evaluate fitness f(x) of each chromosome in
2 NEW POPULATION
0 SELECTION : Based on f(x)
1 RECOMBINATION : Cross-over chromosomes
2 MUTATION : Mutate chromosomes
3 ACCEPTATION : Reject or accept new one
3 REPLACE : Replace old with new population: the new
4 TEST : Test problem criterium
5 LOOP : Continue step 1 – 4 until criterium is
GA vs. Specialized Alg.
Genetic Algorithms (GAs)
Specialized algorithms – best performance for special problems
Genetic algorithms – good performance over a wide range of problems
Guided random search technique
Uses the payoff function to guide
Function Optimization Algorithms?
They are fundamentally the same!!
Things needed for GAs
How do we represent individuals? Domain
How do we interpret individuals?
What is the fitness function?
How are individual chosen for reproduction?
Choose better individuals (probabilistic)
How do individuals reproduce?
Crossover, Mutation, etc.
How is the next generation generated?
Replace badly performing individuals
Binary Encoding/Ternary Encoding
Chromosome A 10110010110011100101
Chromosome B 11111110000000011111
Permutation Encoding (TSP)
Chromosome A 1 5 3 2 6 4 7 9 8
Chromosome B 8 5 6 7 2 3 1 4 9
Real numbers, etc. Specialized
Chromosome 1.235 5.323 0.454 2.321 2.454
Chromosome (left), (back), (left), (right), (forward)
A fitness function quantifies the optimality of a
solution (chromosome) so that that particular
solution may be ranked against all the other
A fitness value is assigned to each solution depending
on how close it actually is to solving the problem.
Ideal fitness function correlates closely to goal +
Example. In TSP, f(x) is sum of distances between
the cities in solution. The lesser the value, the fitter
the solution is
The process that determines which
solutions are to be preserved and allowed
to reproduce and which ones deserve to
The primary objective of the recombination
operator is to emphasize the good solutions and
eliminate the bad solutions in a population, while
keeping the population size constant.
“Selects The Best, Discards The Rest”.
Roulette Wheel Selection
Chromosome # Fitness
2 15.4091 3
Strings that are fitter are assigned a larger slot and hence
have a better chance of appearing in the new population.
GA in Action for 8-Queen
Fitness for 8-Queen?
Minimum conflict fitness function.
Theory (Schema Theorem)
Substring where some positions left
Instance of this schema: 24613587
Theorem: if the average of the instances
the schema is above the mean fitness of
the population, the number of instances of
the schema will increase over time.
VLSI, TSP, Function Optimization, Data
mining, security, etc.