Artificial Immune Systems: An Overview
Jon Timmis
Computing Laboratory
University of Kent at Canterbury CT2 7NF. UK. J.Timmis@kent.ac.uk
http:/www.cs.kent.ac.uk/~jt6
AIS – March 2004 – 1
What do I want to achieve?
Give you a taster of what AIS is all about
Why do we find the immune system useful? Explain what AIS are Show you where they are being used Comments for the future
Talk about all areas of AIS and applications Talk too much about how AIS relate to other bioinspired ideas (although I will mention it)
I won’t:
Go into too much detail: this is an introduction
AIS – March 2004 – 2
Outline
What
are AIS? immunology about AIS Areas
Useful
Thinking The
Application
Future
AIS – March 2004 – 3
Why the Immune System?
Recognition
Anomaly detection Noise tolerance
Robustness Changing nature of self Diversity, Adaptive
Reinforcement learning Memory; Dynamically changing coverage Distributed
Multi-layered
AIS – March 2004 – 4
A Definition
AIS are adaptive systems inspired by theoretical immunology and observed immune functions, principles and models, which are applied to complex problem domains
AIS – March 2004 – 5
Some History
Developed from the field of theoretical immunology in the mid 1980’s.
Suggested
we ‘might look’ at the IS
1990 – Bersini first use of immune algos to solve problems Forrest et al – Computer Security mid 1990’s Hunt et al, mid 1990’s – Machine learning
AIS – March 2004 – 6
Scope of AIS:
Computer Security(Forrest’94’96’98, Kephart’94,
Lamont’98’01,02, Dasgupta’99’01, Bentley’00’01,02) Anomaly Detection (Dasgupta’96’01’02) Fault Diagnosis (Ishida’92’93, Ishiguro’94) Data Mining & Retrieval (Hunt’95’96, Timmis’99’ 01, ’02) Pattern Recognition (Forrest’93, Gibert’94, de Castro ’02) Adaptive Control (Bersini’91)
AIS – March 2004 – 7
Scope of AIS (Cont……):
Job shop Scheduling (Hart’98, ’01, ’02) Chemical Pattern Recognition (Dasgupta’99) Robotics (Ishiguro’96’97,Singh’01) Optimization (DeCastro’99,Endo’98, de Castro ’02) Web Mining (Nasaroui’02) Fault Tolerance (Tyrrell, ’01, ’02, Timmis ’02) Autonomous Systems (Varela’92,Ishiguro’96) Engineering Design Optimization (Hajela’96 ’98,
Nunes’00)
And so on …
AIS – March 2004 – 8
Outline
What
are AIS? immunology about AIS Areas
Useful
Thinking The
Application
Future
AIS – March 2004 – 9
How does it work: A simplistic view
AIS – March 2004 – 10
Learning
Primary Response
Antibody Concentration
Secondary Response
Cross-Reactive Response
Lag Lag
Lag
Response to Ag1 Response to Ag1
...
Response to Ag2
Response to Ag1 + Ag3
... ...
Antigen Ag1 Antigens Ag1, Ag2
...
Antigen Ag1 + Ag3 Time
AIS – March 2004 – 11
Immune Network Theory
Idiotypic B
network (Jerne, 1974)
cells co-stimulate each other
Treat each other a bit like antigens
an immunological memory
Creates
Paratope
Suppression Negative response Ag 1 Idiotope 2 3
Antibody Activation Positive response
AIS – March 2004 – 12
Danger Theory
Proposed by Polly Matzinger, around 1995
Problem: Traditional self/non-self theory doesn’t always match observations
Immune system always responds to non-self
…apart from the nonself it doesn’t respond to (harmless foreign)
Immune system always tolerates self
…apart from the self it doesn’t tolerate (dangerous self)
T-cell activation by APCs
AIS – March 2004 – 13
Immune System: Summary
The host has to distinguish either between self/non-self or dangerous/non-dangerous
When an entity is recognised as foreign (or dangerous)- activate several defense mechanisms leading to its destruction (or neutralization).
Subsequent exposure to similar entity results in rapid immune response.
Overall behavior of the immune system is an emergent property of many local interactions.
So it is useful?
AIS – March 2004 – 14
Outline
What
are AIS? immunology about AIS Areas
Useful
Thinking The
Application
Future
AIS – March 2004 – 15
AIS ..
Think about the use of AIS in terms of:
Representation; Affinity Immune
Measures; Algorithms;
…and how we apply them and to what we apply them Think about the bias of all of the above and what affect that may have on the result (if any).
AIS – March 2004 – 16
Basic Immune Algorithms
Population
based:
Negative Selection Algorithms Clonal Selection Algorithm Bone Marrow Algorithms based Immune Network Algorithms
Network
AIS – March 2004 – 17
Outline
What
are AIS? immunology about AIS Areas
Useful
Thinking The
Application
Future
AIS – March 2004 – 18
Data mining
More
benchmark problem in this case a set of labelled vectors
Assume
Classification
AIS – March 2004 – 19
AIRS: (Artificial Immune Recognition System)
Clonal
Selection of cells: hypermutation binding
Survival Somatic
Resource allocation
Antibody/antigen
AIS – March 2004 – 20
AIRS: Mapping from IS to AIS
Antibody Recognition Feature Vector, Class Combination of feature vector and vector class
Antigens Immune Memory
Training Data Memory cells—set of mutated ARBs
AIS – March 2004 – 21
AIRS Algorithm
Data normalisation and initialisation Memory cell identification and ARB generation Competition for resources in the development of a candidate memory cell Potential introduction of the candidate memory cell into the set of established memory cells
AIS – March 2004 – 22
Classification Accuracy
Important
to maintain accuracy
AIRS1: Accuracy
AIRS2: Accuracy
Iris
96.7
96.0
Ionosphere
94.9
95.6
Diabetes
74.1
74.2
Sonar
84.0
84.9
AIS – March 2004 – 23
Features
No
need to know best architecture to get good results settings within a few percent of the best it
Default
can get
parameters optimize performance for a given problem set
Generalization
User-adjustable
and data reduction
AIS – March 2004 – 24
Unsupervised Learning
Again, We
a benchmark problem in this case a set of unlabelled vectors
Assume
can ask the questions: Is there a large amount of redundancy? Are there any groups or subgroups intrinsic to the data? What is the structural or spatial distribution?
AIS – March 2004 – 25
MARIA: Immune principles employed
B-cells
(antibodies) binding
Antigens Antibody/antigen Clonal Multi-layered
selection process system
AIS – March 2004 – 26
Data mining: Clustering (MARIA)
Limited visualisation (3-d only) Good identification of clusters High compression rate Use vector quantisation to assess classification accuracy
Comparable with SOFM, but less parameters and no need to define structure
AIS – March 2004 – 27
Fun Immune Inspired Ideas at Kent
Immunised Fault Tolerance for Mechatronic devices Exploratory industrial sponsored work Increase availability of machines Prediction of machine states Network wide immune system that is adaptable and can disseminate (or vaccinate) other devices Immune Algorithms for Immunoinformatics Prediction of the binding of T-cells and proteins Very hard problem Using AIRS to do this, so far, quite good. Will try and exploit the nature of the problem to solve it!
AIS – March 2004 – 28
More fun …
Danger theory and Web content mining Extracting information from the content of web pages. Little AIS research in this area. A very large and very dynamic data set. Domain characteristics include: Highly volatile data. High volume of data. Ever-changing content. Need for continuous adaptation.
AIS – March 2004 – 29
Mutation Testing (Motivation)
Variability
in software engineering:
Programming languages Paradigms Programming teams needs testing
Software Test
suites wear out
AIS – March 2004 – 30
Mutation Testing Fundamentals
Programmer Hypothesis: Competent programmers make small coding errors Coupling Effect: Large coding errors are combinations of smaller errors
a program P is incorrect, the correct version differs by only a small amount For all mutations M of P, find test data that either: Causes P to behave incorrectly Distinguishes M from P
If
Competent
AIS – March 2004 – 31
Problems with Traditional Mutation Testing
Computationally
expensive:
Mutants
Large numbers Traditionally interpretive execution
Difficult
to automate completely:
Generating test data Detecting equivalent mutants
AIS – March 2004 – 32
General Aims
studying the operation of a programming team over time:
By
Identify program mutation operators that frequently produce live mutants
Live mutants may reveal common errors
Derive more discriminating test data
AIS – March 2004 – 33
The “Vaccine”
set of test cases that kill the largest proportion of mutants
Smallest
More discriminating data
set of mutation operators that deliver the largest set of consistently living mutants
Smallest
Overcoming common errors
AIS – March 2004 – 34
Web Mining (AISEC)
A study of performance and characteristics of an AIS applied to text mining in a dynamic domain User behaviour and interaction with e-mail can be similar to web pages Supervised classification algorithm E-mail classified as interesting and uninteresting Uses constant(ish) feedback from user Capable of continuous adaptation This tracks concept drift and can also handle
A specialised AIS algorithm based in part on the immune principle of clonal selection No previously documented algorithm was suited for use in this situation without extensive changes
AIS – March 2004 – 35
concept shift
AISEC (2)
AISEC has produced promising results and appears robust Interesting note: Typical accuracy similar to published results from other AIS for text classification (both traditional and continuous learning) Use a larger training set and optimise (the many) parameters Detect when there are the optimum number of cells Future work We are researching adaptive systems for retrieval of interesting information, not necessarily purely accurate information AISEC has been useful providing some evidence AIS applied to this domain would be possible
AIS – March 2004 – 36
Outline
What
are AIS? immunology about AIS Areas
Useful
Thinking The
Application
Future
AIS – March 2004 – 37
The Future
Rapidly emerging field Much work is very diverse More formal approach required? Wide possible application domains What is it that makes the immune system unique? More work with immunologists Theories such as Danger theory, Self-Assertion may have something to say to AIS
AIS – March 2004 – 38
Integrating: Homeostasis
Ability of an organism to achieve a steady state of internal body function in a varying environment
Lots of complex interactions
Nervous system Endocrine system Immune System
Developed a simple neural network and endocrine controller for a mobile robot
AIS – March 2004 – 39
The Future (2)
ARTIST: A Network for Artificial Immune Systems (EPSRC funded network) http://www.artificial-immune-systems.org Work towards: A theoretical foundation for AIS as a new CI Extraction of accurate metaphors Immune System Modelling Application of AIS Train PhD students Fund workshops/meetings Coordinate and Disseminate UK based AIS research (links to Europe)
AIS – March 2004 – 40
AIS Resources: Books
Artificial Immune Systems and Their Applications by Dipankar Dasgupta (Editor) Springer Verlag, January 1999.
Artificial Immune Systems: A New Computational Intelligence Approach by Leandro N. de Castro, Jonathan Timmis, Springer Verlag, November 2002. Immunocomputing: Principles and Applications by Alexander O. Tarakanov, Victor A. Skormin, Svetlana P. Sokolova, Springer Verlag, April 2003.
AIS – March 2004 – 41