An anti-Hebbian model of familiarity discrimination in the perirhinal cortex Malcolm W. Brown Rafal Bogacz University of Bristol Princeton University Perirhinal cortex Storage capacity Reason for capacity difference Work in monkeys has established that discrimination of the relative familiarity or novelty of visual When it is assumed that responses of neurons providing input to the familiarity discrimination network The difference in capacities of the models based on Hebbian and anti-Hebbian learning may be stimuli is dependent on the perirhinal cortex, and this finding is consistent with studies of amnesic are uncorrelated, models based on either Hebbian or anti-Hebbian learning achieve very high storage explained intuitively by the fact that the Hebbian models have a natural tendency to extract features; patients [5,6]. Within the monkey’s perirhinal cortex, ~25% of neurons respond strongly to the sight of capacity, which capacity is much greater than that of associative memories for recall. The former is hence they focus on elements common to all the input patterns (i.e. features). By contrast, the anti- novel objects but respond only weakly or briefly when these objects are seen again [6,14]. proportional to the number of synapses in the network, the latter to that number divided by the number Hebbian model focuses on elements characteristic to individual patterns rather than their common of neurons . This difference may be intuitively explained by comparing the two tasks: recall – for features 30 spikes/s example, you see a person and you want to recall his/her name and the episode of the previous Monkey brain meeting with the person – and familiarity discrimination – you see a person and you want to determine • Hebbian: learns features common to all stimuli Stimulus s whether or not you have seen him/her previously. In the first case, the network has to recall the whole ventral view representation of the name and the episode, which is encoded in the activity of a number of neurons – Novel let us denote this number by N. By contrast, for familiarity discrimination, there is just a binary output: the stimulus is novel or familiar. The number of outputs in the case of familiarity discrimination is N times smaller (so, in this sense, familiarity discrimination is N times easier than recall). Therefore, intuitively, the capacity for familiarity discrimination is of order N times higher. • Retrieval • Familiarity discrimination • Anti-Hebbian: ignores features common to all stimuli Familiar J u Novel l ? i Familiar a • Capacity ~ number of synapses per • Capacity = 0.023 x number of Estimation of the capacity of human perirhinal net neuron  synapses Models of familiarity discrimination • CA3: • Perirhinal cortex: We also estimate the capacity of putative networks of novelty neurons in the human perirhinal cortex: – 2.3*106 pyramidal neurons – 4*106 novelty neurons All previously published models of familiarity discrimination in the perirhinal cortex [1,10,11] are based • Assumptions: on Hebbian learning. Here we present a model based on anti-Hebbian learning. – 4*104 synapses per neuron – 104 synapses per neuron – 4,000,000 novelty neurons The Figure below shows the synaptic plastic changes of a single perirhinal novelty neuron for the anti- – 105 stimuli – 109 stimuli Hebbian model after presentation of a novel stimulus. – No noise – 10,000 synapses per neuron 1,000 synapses per neuron Under the assumption of uncorrelated firing rates of the input neurons, if the perirhinal cortex worked 10,000,000,000 10,000,000,000 akin to these models, it alone could discriminate the familiarity of many more stimuli than current 1,000,000,000 1,000,000,000 neural network models indicate could be recalled (recollected) by all the remaining areas of the 100,000,000 100,000,000 novel cerebral cortex. . 10,000,000 10,000,000 P P 1,000,000 1,000,000 100,000 100,000 Hebbian Neuronal recordings indicate that the perirhinal neurons have correlated firing rates  (i.e. two 10,000 Anti-Hebbian 10,000 1,000 Homosynaptic LTD – known to exist in the perirhinal 1,000 neurons may selectively respond to similar groups of stimuli). Correlation between responses 100 100 cortex  decreases the capacity of different familiarity discrimination models differently. 10 10 1 The left side of the Figure below shows that the capacity of a model  based on Hebbian learning (the 0.000001 0.00001 0.0001 0.001 0.01 0.1 1 1 0.000001 0.00001 0.0001 0.001 0.01 0.1 1 familiar simulations are described in ) decreases very markedly even when the correlation is very small. The correlation correlation right side of the Figure shows that correlation reduces the capacity of the anti-Hebbian model far less than the network based on Hebbian learning. correlation between responses of distant perirhinal To maintain the neuron’s overall excitability, the synaptic Furthermore, for familiarity discrimination networks based on Hebbian learning the influence on neurons measured in  weights of connections from inactive input neurons must capacity of correlation between responses of input neurons increases when the size of the network be increased. This increase may be mediated by grows. By contrast, for the anti-Hebbian model the effect of correlation on capacity decreases with homeostatic mechanisms that act to maintain average neuronal activity and thus promote network stability (they increasing network size. Hence for large networks, the anti-Hebbian model achieves a capacity much Conclusion greater than any of the networks based on Hebbian learning when there are even very small have been reported in cultures and slices of cortical correlations between the responses of the input neurons. neurons; for review see ). If perirhinal cortex worked akin to the anti-Hebbian model, it could discriminate familiarity for up to thousands of times more stimuli than if it worked according to the models based on Hebbian learning. 2500 1400 Note that when the same stimulus is presented again, the membrane potential of the novelty neuron N=100 simulation N=100 sim. The consistency of the anti-Hebbian and other models with the results of experimental observations is will be lower (because the weights of synapses of inputs that were active for this stimulus have been N=100 prediction 1200 N=200 sim. 2000 compared in . reduced) and the novelty neuron will be inactive (or, more generally, less active). Thus the neuron N=200 simulation N=300 sim. 1000 N=100 pred. responds more strongly to novel than familiar stimuli. N=200 prediction The anti-Hebbian model includes a single layer of novelty neurons receiving projections from the input 1500 N=300 simulation N=200 pred. 800 References P neurons. If each novelty neuron makes its own decision about stimulus familiarity, the overall response N=300 prediction N=300 pred. P (“answer”) of the network is encoded in the population activity of the novelty neurons. It is necessary to 600 1000 ensure that individual novelty neurons remain independent assessors of familiarity if the information  R. Bogacz, M.W. Brown and C. Giraud-Carrier, Model of familiarity discrimination in the perirhinal cortex, J. Comp. Neurosci. 10 (2001) 5-23.  R. Bogacz, Computational models of familiarity discrimination in the perirhinal cortex, Ph.D. thesis, University of Bristol, 2001. (also available at: storage capacity of the network is to be maximised . Otherwise, should all the novelty neurons be 400 http://www.math.princeton.edu/~rbogacz).  R. Bogacz and M.W. Brown, Comparison of computational models of familiarity discrimination in the perirhinal cortex, Hippocampus (in press). (also available at: active after the presentation of each of a series of novel stimuli, then the synaptic weights of each of 500 http://www.math.princeton.edu/~rbogacz). the novelty neurons would be modified in the same way, and hence all the novelty neurons would 200  R. Bogacz and M.W. Brown, The restricted influence of the sparseness of coding on the capacity of the familiarity discrimination networks, submitted.  M.W. Brown and J.P. Aggleton, Recognition memory: what are the roles of the perirhinal cortex and hippocampus? Nat. Rev. Neurosci. 2 (2001) 51-62. come to have highly correlated weights. Thus, eventually, they would all be active or inactive together 0  M.W. Brown and J.Z. Xiang, Recognition memory: Neuronal substrates of the judgement of prior occurrence, Prog. Neurobiol. 55 (1998) 149-189. 0  K. Cho, N. Kemp, J. Noel, J.P. Aggleton, M.W. Brown, Z.I. Bashir, A new form of long-term depression in the perirhinal cortex, Nat. Neurosci. 3 (2000) 150-156. and the whole network would have the same capacity as a single novelty neuron. To avoid this 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1  C.A. Erickson, B. Jagadeesh, R. Desimone, Clustering of perirhinal neurons with similar properties following visual experience in adult monkey, Nat. Neurosci. 3 (2000) problem, the number of novelty neurons active for any one stimulus must be limited, i.e. only a subset 1143-1148. of novelty neurons must respond to any given stimulus. This limitation of the number of active novelty sqrt(|rij |) sqrt(|rij |)  J.J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, Proc. Nat. Acad. Sci. 79 (1982) 2554-2558.  K.A. Norman and R.C. O’Reilly, Modelling hippocampal and neocortical contributions to recognition memory: a complementary learning systems approach, Technical Report 01-02, University of Colorado, Boulder, 2001. neurons is achieved in the model by inhibitory competition: only the fraction of neurons with the highest  V.S. Sohal and M.E. Hasselmo, A model for experience-dependent changes in the responses of infero-temporal neurons, Network 11 (2000) 169-190. membrane potentials are selected to be active, the activity of the remainder being suppressed by 1 1 0.185 N 3r 3 3  A. Treves and E.T. Rolls A computational analysis of the role of the hippocampus in memory. Hippocampus 4 (1994) 374-391. inhibition, and only these most active neurons have their weights modified . Pm ax Pmax 0.013N 0.31N 2 r 2  G.G. Turrigiano and S.B. Nelson, Hebb and homeostasis in neuronal plasticity, Cur. Opin. Neurobiol. 10 (2000) 358-364.  J.Z. Xiang and M.W. Brown, Differential neuronal encoding of novelty, familiarity and recency in regions of the anterior temporal lobe, Neuropharmacology 37 (1998) 4 Nr 3 657-676.