David Lamb Introductory Presentation D.Lamb@2005.ljmu.ac.uk Room 608, ext 2280 http://www.staff.ljmu.ac.uk/cmpdlamb Introduction Overview of research to date Cognitive Immunity Subsystems Detection – Machine Learning techniques Danger Theory Novelty Detection Chance Discovery Anticipatory Learning Classifier Systems The “Research Problem” Planned Future Research Cognitive Immunity A Cognitive Immune system should: Identify problems: both vulnerabilities and newly-introduced problems Diagnose the cause and severity of problems… Then, ideally, restore the system to correct functionality Cognitive Immunity: Subsystems A proposed layered model of CI subsystems: Detection – detects events in environment Diagnosis – maps/combines events to form situations Planning – plans actions to resolve a situation Enactment – assesses impact of actions, and performs them Learning/Evolution – evolves the system based on feedback from environment Self-Organisation – re-organises the system to avoid vulnerabilities Current Research Research thus far has been in the area of Cognitive Immunity: Detection and Diagnosis subsystems Tried to concentrate on mechanisms that can provide services suitable for Detection Also looked at existing cognitive software / system models to gain insight into the design of this type of software The following slides aim to present an overview of this research… Novelty Detection: Introduction Novelty Detection systems are concerned with: Detecting the data in a given set of inputs that may be considered abnormal – or novel. Effectively generalising the known type – i.e. not just a pattern match! Useful to Immune Systems as a detector – a known vs. unknown discriminator, allowing a response to unknown data or signals. Novelty Detection: Statistics Traditional Mathematical / Statistical approaches can determine novelty by: Plotting all known data, according to its defining attributes, in an n-dimensional space. Identifying clusters of plots as known types or classes Identifying plots outside of these clusters as novel, abnormal, or unrecognised This approach is complicated by: High-dimensional data Outliers (standalone plots outside a cluster) – both legitimate and those as a result of noisy data Quality of known data samples Novelty Detection: Neural Networks 1 Trained Neural Networks can be used as Novelty Detectors Training a Neural Network – an overview: Standard Multi-Layer Perceptron Neural Networks can be trained to produce certain outputs for certain inputs. This is achieved by repeatedly presenting the network with sample input data, and appropriate target output data. After many training cycles, the network will reproduce the target output data for the specified inputs If the network inputs described the data well and the training data varied sufficiently, the network should perform well on data similar to the training data Novelty Detection: Neural Networks 2 This allows MLP networks to behave as data classifiers: Training data is comprised of samples of known types and suitable output class indication A high signal on a particular network output indicates a particular class/type has been presented as input Confidence scores can be added to quantify the confidence in any given classification However, using classification networks for novelty detection poses the following problems: Accurate classification clearly depends on good quality training data Expensive to retrain to recognise additional classes - typically a full retraining from scratch is required May result in confusion at outputs when presented with truly novel data Novelty Detection: SO(F)Ms Self Organising (Feature) Maps: A special type of neural network that undergoes unsupervised training i.e. the network is trained solely on input data, and doesn’t require additional target output data Can easily derive classes (clusters) of data based on the variety in the input set SOM visualisations are particularly appropriate for presenting high dimensional data in a 2D map output Novelty Detection: Other Approaches Detectors / Selection approaches Known data is coded, typically as binary strings A set of random detectors are created as strings The random detectors are tested on the known/self data Those that match (against self) are eliminated Evolutionary approaches (genetic algorithms) Rules to match known/unknown are coded in strings Generations are evolved based on fitness of previous “parents” and modification via evolution operators: Mutation – one or more bits are changed Combination/Crossover – x bits from one parent, y bits from other parent Novelty Detection and Classification In addition to differentiating between known and unknown, several of the proposed novelty detectors can also serve as advanced classifiers Classification can also prove useful to the Detection layer of an Immune System An ideal data classifier should: Generalise classes (or types) of inputs Operate at a higher, more abstract level than a simple pattern match: (i.e. not just X = Y, but X is similar to Y) The Danger Theory: Introduction Originated in biological immunology Changes emphasis of response to a specified Danger Signal, rather than reacting purely to non-self Can provide a localised response (within the Danger Zone) Simple, (mostly) independent interactions repeated on a large scale produce the desired immune response The Danger Theory: Biological Model Response to Danger Signal (in the illustrated case, cell damage) triggers antibody reaction within the Danger Zone Matching antibodies are then duplicated to facilitate more antigen matching The Danger Theory in Software Using the Danger Theory model in software presents some problems: Representation of Danger Signal(s) Representation of spatial Danger Zone How to implement antibody / antigen recognition How to implement antibody suppression of antigens Chance Discovery: Introduction New-ish field, some disagreement on definition and application: Some argue it is simply a variation on existing data mining themes Broad Definition, “Discovers valuable chance events – those that are rare, but important” Chance Discovery: Overview A Chance Discovery system must be able to perform two main tasks: Identification/Prediction of Chance Events Identification/Prediction of Consequences: Identifying consequences – e.g. associate cause with effect, based on system history. Find the value (or cost) of the cause. Prediction of consequences – where history is not available or inappropriate, prediction with bounded accuracy Chance Discovery: Current Systems Despite the fact that the field is quite new, some prototype/research CD systems exist: Key Graphs A method initially created to index documents Clusters co-occurrences of terms in a document These clusters should indicate topics Index terms are then chosen based on their relationship to other clusters Chance events (i.e. index terms) are chosen based on their links to significant high-frequency events (term clusters). Chance Discovery: Current Systems Knowledge Base – (Change-based CD) World knowledge is modelled as rules in a KB Chance discoveries are made as this knowledge is changed, based on the fact that changes may: Enable/disable some goal(s) Alter the cost/reward of achieving some goal(s) Dialogue approach Dialogue facilitates communication between separate knowledge bases Can be viewed as a distributed extension of the KB approach, deals with separate (and possibly differing) KBs (Anticipatory) Learning Classifier Systems ALCS – Currently research in progress! ALCS are cognitive systems that form anticipations about future events based on current behaviour and observations They are of interest, as they may represent a significant building block towards a CI system model… ALCS: Components Two essential ALCS components: ALP – Anticipatory Learning Process – compares anticipations with actual results, resulting in specialised rules that describe the observed behaviour. Genetic Generalisation Mechanism – Generalises accurate rules from the over- specified ALP output, making the model more compact The “Research Problem” How to create a reusable model for Cognitive Immunity, and to find components suitable for use in that model How to apply that model to a real-world example How to implement a complete system, using the proposed subsystems for a real- world problem Planned Research How do I get from where I am now to where I want to be? Which significant areas of research must be covered? A continuation of ALCS research, plus… Research into other types of Cognitive Systems and Artificial Immune Systems to understand the various ways of modelling these systems Research into more components or services suitable for the identified CI subsystems The End Thanks for listening!! Any questions?
Pages to are hidden for
"David Lamb Introductory Presentation"Please download to view full document