Artificial Immune Systems An Overview

Artificial Immune Systems: An Overview Jon Timmis Computing Laboratory University of Kent at Canterbury CT2 7NF. UK. J.Timmis@kent.ac.uk http:/www.cs.kent.ac.uk/~jt6 AIS – March 2004 – 1 What do I want to achieve?  Give you a taster of what AIS is all about  Why do we find the immune system useful?     Explain what AIS are Show you where they are being used Comments for the future Talk about all areas of AIS and applications Talk too much about how AIS relate to other bioinspired ideas (although I will mention it) Go into too much detail: this is an introduction I won’t:    AIS – March 2004 – 2 Outline What Useful are AIS? immunology Thinking The about AIS Areas Application Future AIS – March 2004 – 3 Why the Immune System?  Recognition   Anomaly detection Noise tolerance        Robustness Changing nature of self Diversity, Adaptive Reinforcement learning Memory; Dynamically changing coverage Distributed Multi-layered AIS – March 2004 – 4 A Definition AIS are adaptive systems inspired by theoretical immunology and observed immune functions, principles and models, which are applied to complex problem domains AIS – March 2004 – 5 Some History  Developed from the field of theoretical immunology in the mid 1980’s.  Suggested we ‘might look’ at the IS  1990 – Bersini first use of immune algos to solve problems Forrest et al – Computer Security mid 1990’s   Hunt et al, mid 1990’s – Machine learning AIS – March 2004 – 6 Scope of AIS:  Computer Security(Forrest’94’96’98, Kephart’94,      Lamont’98’01,02, Dasgupta’99’01, Bentley’00’01,02) Anomaly Detection (Dasgupta’96’01’02) Fault Diagnosis (Ishida’92’93, Ishiguro’94) Data Mining & Retrieval (Hunt’95’96, Timmis’99’ 01, ’02) Pattern Recognition (Forrest’93, Gibert’94, de Castro ’02) Adaptive Control (Bersini’91) AIS – March 2004 – 7 Scope of AIS (Cont……):         Job shop Scheduling (Hart’98, ’01, ’02) Chemical Pattern Recognition (Dasgupta’99) Robotics (Ishiguro’96’97,Singh’01) Optimization (DeCastro’99,Endo’98, de Castro ’02) Web Mining (Nasaroui’02) Fault Tolerance (Tyrrell, ’01, ’02, Timmis ’02) Autonomous Systems (Varela’92,Ishiguro’96) Engineering Design Optimization (Hajela’96 ’98, Nunes’00)  And so on … AIS – March 2004 – 8 Outline What are AIS? immunology Useful Thinking The about AIS Areas Application Future AIS – March 2004 – 9 How does it work: A simplistic view AIS – March 2004 – 10 Learning Primary Response Antibody Concentration Secondary Response Cross-Reactive Response Lag Lag Lag Response to Ag1 Response to Ag1 ... Response to Ag2 Response to Ag1 + Ag3 ... ... Antigen Ag1 Antigens Ag1, Ag2 ... Antigen Ag1 + Ag3 Time AIS – March 2004 – 11 Immune Network Theory Idiotypic B network (Jerne, 1974) cells co-stimulate each other  Treat each other a bit like antigens an immunological memory Creates Paratope Suppression Negative response Ag 1 Idiotope 2 3 Antibody Activation Positive response AIS – March 2004 – 12 Danger Theory Proposed by Polly Matzinger, around 1995 Problem: Traditional self/non-self theory doesn’t always match observations    Immune system always responds to non-self …apart from the nonself it doesn’t respond to (harmless foreign)  Immune system always tolerates self …apart from the self it doesn’t tolerate (dangerous self)  T-cell activation by APCs AIS – March 2004 – 13 Immune System: Summary  The host has to distinguish either between self/non-self or dangerous/non-dangerous When an entity is recognised as foreign (or dangerous)- activate several defense mechanisms leading to its destruction (or neutralization). Subsequent exposure to similar entity results in rapid immune response. Overall behavior of the immune system is an emergent property of many local interactions. So it is useful?     AIS – March 2004 – 14 Outline What are AIS? immunology Useful Thinking The about AIS Areas Application Future AIS – March 2004 – 15 AIS ..  Think about the use of AIS in terms of:  Representation;  Affinity  Immune Measures; Algorithms;   …and how we apply them and to what we apply them Think about the bias of all of the above and what affect that may have on the result (if any). AIS – March 2004 – 16 Basic Immune Algorithms Population    based: Negative Selection Algorithms Clonal Selection Algorithm Bone Marrow Algorithms based Immune Network Algorithms Network  AIS – March 2004 – 17 Outline What are AIS? immunology Useful Thinking The about AIS Areas Application Future AIS – March 2004 – 18 Data mining More benchmark problem in this case a set of labelled vectors Assume Classification AIS – March 2004 – 19 AIRS: (Artificial Immune Recognition System) Clonal Selection of cells: hypermutation binding Survival  Somatic Resource allocation Antibody/antigen AIS – March 2004 – 20 AIRS: Mapping from IS to AIS Antibody Recognition Feature Vector, Class Combination of feature vector and vector class Antigens Immune Memory Training Data Memory cells—set of mutated ARBs AIS – March 2004 – 21 AIRS Algorithm    Data normalisation and initialisation Memory cell identification and ARB generation Competition for resources in the development of a candidate memory cell Potential introduction of the candidate memory cell into the set of established memory cells  AIS – March 2004 – 22 Classification Accuracy Important to maintain accuracy AIRS1: Accuracy AIRS2: Accuracy Iris 96.7 96.0 Ionosphere 94.9 95.6 Diabetes 74.1 74.2 Sonar 84.0 84.9 AIS – March 2004 – 23 Features No need to know best architecture to get good results settings within a few percent of the best it Default can get parameters optimize performance for a given problem set Generalization User-adjustable and data reduction AIS – March 2004 – 24 Unsupervised Learning Again, We   a benchmark problem in this case a set of unlabelled vectors Assume can ask the questions: Is there a large amount of redundancy? Are there any groups or subgroups intrinsic to the data? What is the structural or spatial distribution?  AIS – March 2004 – 25 MARIA: Immune principles employed B-cells (antibodies) Antigens Antibody/antigen Clonal Multi-layered binding selection process system AIS – March 2004 – 26 Data mining: Clustering (MARIA)     Limited visualisation (3-d only) Good identification of clusters High compression rate Use vector quantisation to assess classification accuracy  Comparable with SOFM, but less parameters and no need to define structure AIS – March 2004 – 27 Fun Immune Inspired Ideas at Kent   Immunised Fault Tolerance for Mechatronic devices  Exploratory industrial sponsored work  Increase availability of machines  Prediction of machine states  Network wide immune system that is adaptable and can disseminate (or vaccinate) other devices Immune Algorithms for Immunoinformatics  Prediction of the binding of T-cells and proteins  Very hard problem  Using AIRS to do this, so far, quite good.  Will try and exploit the nature of the problem to solve it! AIS – March 2004 – 28 More fun …   Danger theory and Web content mining  Extracting information from the content of web pages.  Little AIS research in this area.  A very large and very dynamic data set. Domain characteristics include:  Highly volatile data.  High volume of data.  Ever-changing content.  Need for continuous adaptation. AIS – March 2004 – 29 Mutation Testing (Motivation) Variability    in software engineering: Programming languages Paradigms Programming teams needs testing Software Test suites wear out AIS – March 2004 – 30 Mutation Testing Fundamentals Programmer Hypothesis:  Competent programmers make small coding errors Coupling Effect:  Large coding errors are combinations of smaller errors a program P is incorrect, the correct version differs by only a small amount For all mutations M of P, find test data that either:  Causes P to behave incorrectly  Distinguishes M from P If Competent AIS – March 2004 – 31 Problems with Traditional Mutation Testing Computationally  expensive: Mutants   Large numbers Traditionally interpretive execution Difficult   to automate completely: Generating test data Detecting equivalent mutants AIS – March 2004 – 32 General Aims studying the operation of a programming team over time:  By Identify program mutation operators that frequently produce live mutants  Live mutants may reveal common errors  Derive more discriminating test data AIS – March 2004 – 33 The “Vaccine” set of test cases that kill the largest proportion of mutants  Smallest More discriminating data set of mutation operators that deliver the largest set of consistently living mutants  Smallest Overcoming common errors AIS – March 2004 – 34 Web Mining (AISEC)    A study of performance and characteristics of an AIS applied to text mining in a dynamic domain User behaviour and interaction with e-mail can be similar to web pages Supervised classification algorithm  E-mail classified as interesting and uninteresting  Uses constant(ish) feedback from user  Capable of continuous adaptation  This tracks concept drift and can also handle A specialised AIS algorithm based in part on the immune principle of clonal selection  No previously documented algorithm was suited for use in this situation without extensive changes AIS – March 2004 – 35 concept shift  AISEC (2)  AISEC has produced promising results and appears robust  Interesting note: Typical accuracy similar to published results from other AIS for text classification (both traditional and continuous learning)  Use a larger training set and optimise (the many) parameters  Detect when there are the optimum number of cells  Future work  We are researching adaptive systems for retrieval of interesting information, not necessarily purely accurate information  AISEC has been useful providing some evidence AIS applied to this domain would be possible AIS – March 2004 – 36 Outline What are AIS? immunology Useful Thinking The about AIS Areas Application Future AIS – March 2004 – 37 The Future      Rapidly emerging field Much work is very diverse  More formal approach required? Wide possible application domains What is it that makes the immune system unique? More work with immunologists  Theories such as Danger theory, Self-Assertion may have something to say to AIS AIS – March 2004 – 38 Integrating: Homeostasis Ability of an organism to achieve a steady state of internal body function in a varying environment  Lots of complex interactions    Nervous system Endocrine system Immune System  Developed a simple neural network and endocrine controller for a mobile robot AIS – March 2004 – 39 The Future (2)      ARTIST: A Network for Artificial Immune Systems (EPSRC funded network)  http://www.artificial-immune-systems.org Work towards:  A theoretical foundation for AIS as a new CI  Extraction of accurate metaphors  Immune System Modelling  Application of AIS Train PhD students Fund workshops/meetings Coordinate and Disseminate UK based AIS research (links to Europe) AIS – March 2004 – 40 AIS Resources: Books  Artificial Immune Systems and Their Applications by Dipankar Dasgupta (Editor) Springer Verlag, January 1999. Artificial Immune Systems: A New Computational Intelligence Approach by Leandro N. de Castro, Jonathan Timmis, Springer Verlag, November 2002.   Immunocomputing: Principles and Applications by Alexander O. Tarakanov, Victor A. Skormin, Svetlana P. Sokolova, Springer Verlag, April 2003. AIS – March 2004 – 41