; A Recommender System based on the Immune Network
Documents
Resources
Learning Center
Upload
Plans & pricing Sign in
Sign Out
Your Federal Quarterly Tax Payments are due April 15th Get Help Now >>

A Recommender System based on the Immune Network

VIEWS: 81 PAGES: 38

  • pg 1
									Artificial Immune Systems

A Recommender System based on the click to edit Networktext master Immune

Dr Uwe Aickelin
• Click to edit Master text styles • Second level

• • •

• The Recommendation Problem Third level Fourth level Fifth level • The AIS Approach • Algorithm Walkthrough • Results and Discussion

Artificial Immune Systems

“What movies would you predict/recommend?”

The Recommendation Problem click to edit master text

• Click Prediction styles to edit Master text • Second level

Recommendation
Give me a „top 10‟ list of films I might like

•What level would I give Third rating • Fourth level film? this • Fifth level

Prediction quality can be Recommendation quality assessed by absolute can be assessed by a error ranking „discordance‟ metric

Artificial Immune Systems

click to edit master text The Biological Immune System
Innate
vs

Acquired

• Click to edit Master text styles • Second level vs Cell Mediated • Third level • Fourth level T Cell (CD-4, Helper) • Fifth level Binds to MHC-antigen
complex Secretes cytokines to help…

Humoral

B Cell
Secretes

T Cell (CD-8, Killer)
Kills cell (viruses)

Antibody
which binds to antigen and recruits phagocytes (innate)

How do we protect the body against infection? (Antigens)

Artificial Immune Systems

The Recommendation Problem click to edit master text
EachMovie database User profiles (3M votes 70k users)
• Click to edit Master text styles User level • Second Profile: set of tuples {movie, rating}

• Third level Me: level • Fourth My user profile • Fifth level Neighbour: User profile of someone else

Similarity metric: Correlation score between user profiles
Neighbourhood: Group of neighbours similar to me

Recommendations: generated from neighbourhood

Artificial Immune Systems

clickThe AIS Approach to edit master text
EachMovie database User profiles
• Click to edit Master text styles User level • Second Profile: set of tuples {movie, rating}

• Third level Me: level • Fourth My user profile Antigen • Fifth level Neighbour: User profile of someone else Antibody

Similarity metric: Correlation score between user profiles
Antibody – Antigen Binding Antibody – Antibody Binding

Neighbourhood: Group of neighbours similar to me
Group of antibodies similar to antigen and dissimilar to other antibodies

Recommendations: generated from neighbourhood

Artificial Immune Systems

click to editAlgorithm The AIS master text
Start with empty AIS

• Click to edit Master text styles WHILE (AIS not full) && (More users) • Second level DO

Encode target user as an antigen Ag

• Third level Ab2 Ab1 • Fourth level Ag • Fifth level
Ab3
Ab4
OD

Add next user as an antibody Ab IF (AIS at full size)

Iterate AIS FI

Generate recommendations from AIS

Artificial Immune Systems

Algorithm walkthrough: Encoding click to edit master text
Suppose we have 5 users and 4 movies
• Click to edit Master text styles DATABASE • Second level

u1={(m1 • Third level ,v11),(m2,v12),(m3,v13)} • Fourth level ),(m ,v ),(m ,v ),(m ,v )} u2={(m1,v21 2 22 3 23 4 24 • Fifth level u3={(m1,v31),(m2,v32),(m4,v34)}
u4={(m1,v41),(m4,v44)}

u5={(m1,v51),(m2,v52),(m3,v53), (m4,v54)}
• We do not have user votes for every film

• We want to predict the vote of user u4 on movie m3

Artificial Immune Systems

Algorithm walkthrough (1) click to edit master text
AIS
Start with empty AIS
• Click to edit Master text styles DATABASE • Second level

u1, level • Third u2, u3, u4, u5 • Fourth level •Encode user for whom to make predictions as an antigen Ag Fifth level

DATABASE
u1, u2, u3, u4, u5

u4

AIS Ag

Artificial Immune Systems

Algorithm walkthrough (2) click to edit master text
Add antibodies until AIS is full…
• Click to edituser as text antibody Ab1 Add next Master an styles • Second level

AIS
Ag

• Third level DATABASE • Fourth level u1, u2, u , u , u • Fifth level 3 4 5 Add users 2 and 3 …

u1

Ab1
AIS u2,u3

DATABASE u1, u2, u3, u4, u5

Ag Ab1 Ab2

Ab3

Artificial Immune Systems

Algorithm walkthrough (3) click to edit master text
After some more iterations… the AIS has filled up:

Ab3 • Click to edit Master text styles

Table • Second levelof matching Scores between Ab and Ag
• Fourth level • Fifth level

• Ab1 Third level
Ag Ab2

MS14, MS24, MS34

Table of matching Scores between Antibodies

MS12 = CorrelCoef(Ab1, Ab2)
MS13 = CorrelCoef(Ab1, Ab3) MS23 = CorrelCoef(Ab2, Ab3)

Artificial Immune Systems

Algorithm walkthrough (4) click to edit master text
AIS is now at full size so begin iterations…
•Calculate edit Master text styles for each Ab, considering Click to new CONCENTRATION •interactions with Ag (STIMULATION) and other Ab (SUPPRESSION) Second level

• Third level • Fourth level • Fifth level AIS
Ag Ab1 Ab2 Ag

AIS

Ab3

Ab1 Ab 2 Ab2 Ab1 Ab2 Ab2

Ab2

Notice that antibody 3 has been eliminated.

Artificial Immune Systems

Algorithm walkthrough (5) click to edit master text
If AIS not yet full and more users available, repeat.
• Click to edit Master text styles Otherwise: GENERATE RECOMMENDATION from CONCENTRATION and ANTIGEN Correlation. • Second level

• Third level • Fourth level • Fifth level
Ag
Ab1 Ab1 Ab2

AIS

Ab2 Ab2

Ab2 Ab2

Recommendation for user u4 on movie m3 will be highly based on vote on m3 of user u2

Artificial Immune Systems

click to Results edit master text
• Tested against EachMovie database (15000 users, 1628 films) • Click to edit Master text styles • Second level compared to standard method • Results • Third level (Pearson k-nearest neighbours)
• Fourth level • Prediction : Results of same quality • Fifth level

• Recommendation: Improved results, 4 out of 5 films correct versus 3 out of 5.

Artificial Immune Systems
100 90

1. Stimulation and suppression affect neighbourhood click to edit master text size and number of users looked at
100

Neighbourhood size

90 80 70 60 50 40 30 20 10 0

Rate 0.2 Rate 0.3 Rate 0.5

Neighbourhood

80 70 60 50 40 30 20 10 0 0

• Click to edit Master text styles • Second level

• Third level • Fourth level Stimulation Rate • Fifth level
0.2 0.4 0.6

0.8

1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Suppression rate
16000

15000

Number reviewers

14000 12000 10000 8000 6000 4000 2000 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

# users

10000

5000

Rate 0.2 Rate 0.3 Rate 0.5

0 0 0.2 0.4 0.6 0.8 1

Stimulation Rate

Suppression Rate

Artificial Immune Systems

2. AIS matches Pearson for prediction click to edit master text
1

Mean Absolute Error

0.95 0.9 0.85

AIS (av) SP (av) SP baseline

• Click to edit Master text styles 0.8 0.75 • Second level

• Third level 0.65 0.6 • Fourth level • Fifth 0.55 level
0.5 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.7

Stimulation Rate

Artificial Immune Systems

3. AIS surpasses Pearson for Recommendation click to edit master text
0.55

Relative Recommendation Recommendation accuracy Accuracy (Kendall's Tau)

0.5

AIS (av) SP (av) SP Baseline

• Click to edit Master text styles 0.45 • Second level

• Third level 0.4 120.0% • Fourth level • Fifth 0.35 level
110.0% 0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Stimulation Rate
100.0%

90.0%

Rate 0.2 Rate 0.3 Rate 0.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

80.0%

Suppression rate

Artificial Immune Systems


click to edit master text Evaluation
General purpose recommendation tool (e.g. edit Master text • Click to Bookmarks) styles
• Second level


• Third level examination of AIS dynamics: • Fourth level - Idiotypic effect for more varied population • Fifth level

Collaborative Filtering is a useful vehicle for

- Potential for distribution - Smaller neighbourhoods (vs computational cost)

Wider applicability (e.g. online community formation)


Artificial Immune Systems


Speculation: online community click to edit master text formation
Idiotypic effects alter nature of community
• Click to edit Master text styles  How important is diversity? • Second level
 Are there other network effects that can be used? • Third level (hubs, routers etc) • Fourth level • Fifth level  Distribution: the snowball effect 

What about interacting communities?

areas: ad-hoc community formation, knowledge management, P2P routing…

 Application

Artificial Immune Systems



click to for Security AIS edit master text
Change detection (Checksums)
„Self‟ : files, network traffic, system calls • Click to edit Master text styles  Antibodies • Second level creation: positive vs negative selection  Collaboration between different populations/sites • Third level • Fourth level  Representation: binary string or symbolic (rules) • Fifth levelIS features:  Other

activation thresholds (vs false positives) co-stimulation (vs false positives) memory detectors (secondary response) MHC masks to cover „holes‟ (similar to self)

Artificial Immune Systems

Example: Hofmeyr & Forrest 2000 click to edit master text

• Click to edit Master text styles • Second level

• Third level • Fourth level • Fifth level

Artificial Immune Systems

click to for Security AIS edit master text
Evaluation
 Applied to network styles • Click to edit Master textintrusion, virus detection… • Second level  Good results on test systems • Third level BUT… • Fourth level  Negative Selection doesn‟t scale • Fifth level  Inefficient to map entire non self universe



Changes over time representation of self matching

 Appropriate  Appropriate 

Primary response requires infection?

Artificial Immune Systems

Traditional Selfedit master text click to - Non Self Distinction

• • • • • •

An immune Master text triggered when the body encounters Click to edit response is styles something foreign. Second level The difference between self and non-self is learnt early in life. Third level E.g. eliminate those T- and B-cells that react to self. Fourth level

• Fifth level • Problems: • No reaction to foreign bacteria in gut • No reaction to food we eat • The human body changes over its life • Auto-immune diseases • Tumours / Transplants

Artificial Immune Systems

click to edit master text The Danger Theory

• Click to edit Master text styles should be responded to? Need for discrimination: What • Second level Respond to Danger not to “foreignness”.

• Third level attack everything that is foreign. No need to • Fourth level Danger is measured by damage / distress signals. • Fifth level
Advantages: • Can take care of non-self but harmless • Can take care of self but harmful

Artificial Immune Systems

Danger to edit master text click Model Conclusions

• Click to edit Master text styles useful. Self-Nonself discrimination still • Second does not cause immune response. Nonself level

• Third level Danger Signals trigger immune response. • Fourth level • Fifth level of semantics? A question
• Can this model help us build an AIS for security applications? • What would be „danger signals‟?

Artificial Immune Systems

click to edit master text Discussion

• Click to edit Master text styles • Second level

• Third level • Fourth level • Fifth level Uwe Aickelin:

http://www.aickelin.com/ Steve Cayzer: http://www-uk.hpl.hp.co.uk/people/steve_cayzer/

Artificial Immune Systems

click to edit master text Additional Slides

• Click to edit Master text styles • Second level

• Third level • Fourth level • Fifth level

Artificial Immune Systems

click Models - Idiotypic AIS to edit master text
Antibody
• Click to edit Master text styles • Second level

Farmer et al 1986
• Paratope/Epitopes Lock and Key

• Third level • Fourth level Paratope Antibody • Fifth level
Epitope

Interchangeable?
• Behaviour Matching

Antigen

Idiotypic (Memory, autoimmune)

Artificial Immune Systems

AIS Modelsmaster text click to edit - Idiotypic
Internal Image Anti-Idiotypic Set of Antigen P2 I2 P3 I3 Click to edit Master text styles + Second level Third level P1 Fourth1 level I Fifth level Idiotypic Set

Jerne’s Big Idea (1974)
Idiotype: specificity of antibody (epitopes to which it will bind) Idiotope: An idiotypic epitope

• •

• • •

Antigen

Evidence: Antibodies produced against antibodies of same species (cf individual)

Artificial Immune Systems

AIS Modelsmaster text click to edit - Idiotypic
In Words…
• Click to edit Master text styles hypothesis (Jerne 1974) The idiotypic network • Second builds on the recognition that antibodies can match level other • Third level antibodies as well as antigens. A group of antibodies, which match an antigen, • Fourth level may be matched by other antibodies which may in • Fifth level turn be matched by yet other antibodies. This stimulatory effect will set up activation chains or loops. Matched antibodies are suppressed, and this effect will encourage diversity In Formulae…

Artificial Immune Systems

AIS Modelsmaster text click to edit - Idiotypic
dxii  antigens   antibodies   Iam  antibodies antigens    death   I am dx          c    recognised     recognised   recognised     rate   dt recognised   recognised   recognised   dt           Click to edit Master text styles N n N  Second level m x x  k  c  ji i j 1  mij xi x j   m ji xi y j   k2 xi j 1 j 1  Third level  j 1 mij   G Fourth level  ei n k   p j n   s  1 k Fifth level

• •

• • •







• • •

For N antibodies, n antigens. xi is the concentration of antibody i p and e stand for „paratope‟ and „epitope.‟






s is the matching threshold. G is a rectifier function which outputs 0 for all negative input. k is the allowable overlap

Artificial Immune Systems

Recommendation Approaches click to edit master text
Simple user comparisons (Pearson, cosine, kNearest Neighbour) Click to edit Master text styles curse of dimensionality  Problems: Sparsity, Second  Memory vs Model based approaches level  Transformative and Transitive functions Third level  Default votes, Content based, Learning Fourth level algorithms Fifth level  Challenge of distribution (vs centralization)


• •

• • •

Artificial Immune Systems

System Description: Encoding click to edit master text
Users are represented as a set of tuples which represent their • Click to edit Master text styles votes: • Second level • Third level id1 , score1, id 2 , score 2  id n , score n  User  ... • Fourth level • Fifth level

Artificial Immune Systems

System Description: Matching click to edit master text
We use the Pearson correlation measure • Click to edit Master text styles n • Second level  ui  u vi  v 
r • Third level  n n 2 2  ui  u   vi  v  • Fourth level i 1 i 1 • Fifth level The measure is amended as follows
i 1

if n  0, r  NO _ OVERLAP _ DEFAULT if

ui  u   vi  v 2 
2 i 1 i 1

n

n

 0, r  ZERO _ VARIANCE _ DEFAULT

if n  P, r 

n r P

( where P  overlap penalty )

Artificial Immune Systems

Parameters: Matching Function click to edit master text
Parameter Value Click to edit Master Minimum 5 expected overlap Second level 0.0 Zero Variance Default Third level Use minimum True expected level Fourth overlap No Overlap 0.0 Fifth level Default Use mean of False overlap Comments styles Minimum expected overlap between 2 users (used to calculate penalty) Correlation score when users have overlaps with zero variance Use minimum overlap (i.e. penalise users with less than expected overlap) Correlation score if 2 users have no overlapping items Use mean of overlapping votes only for correlation (otherwise use mean over all votes of user)

• •

text

• • •

Artificial Immune Systems
Parameter Suppression Rate Rate Constant

click to edit master text Parameters: AIS
Comments Suppression constant (weighting on antibodyantibody suppression term) 0.25 for single, 1 Rate constant, applied to each match Click to edit Master text styles Between 0 and 1 for idiotypic calculation. Stimulation Rate 0 Stimulation constant (weighting on antibodySecond level antibody stimulation term) Death Rate 0.1 Death Rate of antibodies (ie % that dies off per Third level unit time) Maximum 100.0 Maximum concentration of antibody or antigen in Fourth level Concentration this AIS Minimum 0 Minimum concentration of antibody or antigen in Fifth level Concentration this AIS Initial 1.0 Initial concentration of antibody or antigen in this Concentration AIS Use True Should we use concentration to weight stimulation Concentration and suppression? Use Absolute True Should we use absolute match score for weighting (hence negative correlations are treated as valuable) Synchronous True Should we apply concentration changes synchronously (in batch) Value 0.001

• •

• • •

Artificial Immune Systems

System Description: Prediction click to edit master text
We predict a rating by using a weighted average over the neighbourhood of a user:

• Click to edit Master text styles • Second level

• Third level • Fourth level  pi  u • Fifth level

vN

 w v w
uv vN

i

 v

uv

wuv  r

(note relative not absolute )

vi  DEFAULT _ VOTE if v has not voted on i

Artificial Immune Systems
Parameter Cluster Size

click to edit master text Parameters: Prediction
Value 50-100 Comments Cluster size (AIS size) Should be >= styles neighbourhood size Do we build AIS from scratch for each prediction or start with one massive AIS? Should we use a default vote for prediction purposes? Neighbourhood size (k-NN parameter). Should be <= cluster size. Use idiotypic immune system (with antibodyantibody interactions) Default vote (if used). Use correlation scores to weight prediction Use category information to help make prediction Max iterations with no change in AIS Weight prediction by antibody concentration (as well as correlation)

• Click to edit Master text Build From BOTH • Scratch Second level

• Votes level Third • Neighbourhood Fourth level size • Use Idiotypic Fifth level AIS
Default Vote Use Correlation Use category Max Iterations Use Concentration

Use Default

True 30-50 BOTH 2.0 True False 5 False

Artificial Immune Systems

System Description: Evaluation click to edit master text
 actual  predicted
n

• Click to edit Master text styles • Mean Absolute Error MAE  • Second level

n
  actual  predicted      n  n    
2

• Third level • Variance Fourth level • Fifth level

 

2  actual  predicted  n

n

Precision
P

vs

Recall
C R U U where R  set of recommenda tions

R U R where R  set of recommenda tions U  items that user liked

U  items that user liked


								
To top