Learning Center
Plans & pricing Sign in
Sign Out


									               Caching and replay of place sequences in a Temporal Restricted Boltzmann Machine model of the hippocampus
               Sue Becker, Dept. of Psychology Neuroscience & Behaviour, McMaster Univ., and Geoff Hinton, Dept. of Computer Science, Univ. of Toronto

1. Summary                                                                           3. Learning in RBMs and TRBMs                                                          5. Training patterns                                   Place tuning in linear track environment
The hippocampus is thought to be critical for the rapid formation and cued           3.1 Restricted Boltzmann Machines
recall of complex event memories. For example, a set of hippocampal
place cells that fires in a particular sequence during exploration may fire in                                                                                              Figure 4: Spatial locations are
the same sequence during sleep [1,2,3]. We propose a novel model of the               Probabilities of states P(V,H) are                                                    coded by Boundary Vector
hippocampal memory system that reconciles a wide range of                             proportional to their Boltzmann                                                       Cells (BVCs) [11]. Each BVC’s
neurobiological data. Here we address the question of how the                         free energies.
hippocampus encodes sequences rapidly, and what is the function of                                                                                                          tuning curve is a product of 2
sequence replay. Our model draws upon two recent developments of the                                                                                                        Gaussians, tuned to distance
                                                                                      In an RBM, conditional probabilities of                                               from and direction to
Restricted Boltzmann Machine (RBM): 1) a hierarchy of sequentially
trained RBM's [4], and 2) the extension to sequential inputs employed in
                                                                                      H & V can be computed efficiently:                                                    environmental boundaries                                Figure 6: Place tuning of several CA units
the Temporal RBM [5]. The top two layers of the model, representing the               P(V , H )  exp(V WH  a V  bH ) / Z
dentate gyrus (DG) and CA fields, are connected via undirected links to                                                                                                                                                            Sequence replay
form an autoassociator, allowing the model to generate memories of                    P( H j  1 | V )   (b j  W V )     '
                                                                                                                          :, j
coherent events, rather than generating top-level unit states
independently. The CA region also has directed connections from previous              P(Vi  1 | H )   (ai  Wi ,: H )
CA states, representing the CA3 recurrent connections. Thus the
probability distribution learned over the visible and hidden units is                  ( x)  1 /(1  exp( x))
conditioned on earlier states of the autoassociator. The model is trained by
contrastive Hebbian learning, with data-driven and generative phases
providing the statistics for the positive and negative Hebbian updates                                                                                                              X
respectively. This is broadly consistent with Hasselmo's proposal that the
hippocampus oscillates between encoding and recall modes within each
theta cycle [6]. When trained on a variety of spatial environments (fig 5),
the hippocampal units develop place fields (fig 6), while the CA time-
delayed recurrent collaterals encode the sequential structure of the data                                                                                                  Figure 5: The model was trained on 3 environments, a
(fig 7). It has been widely assumed that sequence replay is required for                                                                                                   linear track, a U-shaped maze and a rectangular box.
memory consolidation, in which rapidly stored hippocampal memories are                                                                                                     Left: U-maze, with training locations shown by blue
gradually transferred to cortex. Alternatively, the hippocampus may always                                                                                                 dots.
be required to recall complex associative memories, while replay allows
the brain to maintain consistent forward and generative models in the
                                                                                                                                                                           Right: the reconstruction of the boundaries from the
                                                                                                                                                                           BVC representation of the location marked X in the U-        Time
hippocampal-cortical system [7].                                                     3.2 Contrastive divergence learning [10]
                                                                                     Maximizing likelihood requires settling to equilibrium.
2. Data not explained by most HC                                                     Instead, minimize diff. between 2 K.L. divergence terms:
                                                                                                                  ()                                        ()
  models                                                                             KLD( P     (0)
                                                                                                      (d ), P           (d ))  KLD( P       (1)
                                                                                                                                                   (d ), P         (d ))   6. Simulation results
                                                                                     Wi , j  Vi      (0)
                                                                                                              H   (0)
                                                                                                                  j        Vi H
                                                                                                                                 (1)   (1)
                                                                                                                                       j                                                                                            Figure 7: Model’s reconstruction of a sequence of
Anatomy (fig 1)                                                                                                                                                                                                                      spatial locations along the linear track.
• Cascaded                                                                                                                                                                  Input reconstruction
  architecture                                                                       3.3 Temporal Restricted Boltzmann Machines
• Reciprocal                                                                                                                                                                                                                         7. Discussion
  connectivity                                                                        This is one example of a
  btw DG<-->CA3                                                                                                                                                                                                                      The model learns spatial environments gradually, but
                                                                                      class of models called
  and CA<-->EC                                                                                                                                                                                                                         can rapidly cache state sequences in its temporal
                                                                                      TRBM discussed in [5].                                       C1                                                                                  generative model (CA recurrent collaterals)
                       Figure 1. Cascaded architecture of the HC.                                                                                                                                                                    Making the temporal connections bi-directional
Activation                                                                                                                                                                                                                             requires propagation of states backwards in time,
                                                                                                                                                                                                                                       accounting for reverse sequence replay.
                                                                                     P(Vt , H t | H )  exp(Vt t  bH t
                                                                                                       t 1
                                                                                                                      WH                           W                                                                                 The addition of sparseness constraints sharpens
  (fig 2)                                                                                             t m                                                                                                                             place-tuning and leads to a grid-like representation
• Theta vs sharp
                          Figure 2. Left: Theta-modulated gamma                       H t1C1 H t  H t 2 C 2 H t  ...
                          ripples (fig 2A from Chrobak et al 2000) [8].                                                                                                                                                              8. References
                          Right: Sharp waves (fig 1 A, C from Chrobak &               H tm C m H t ) / Z
                          Buszaki, 1994). [9] Reproduced with permission.                                                                                                                                                            [1] Wilson, M.A. and McNaughton, B.L. (1994), Science
Sequence                                                                             4. A TRBM model of the hippocampus                                                                                                                 265(5172):676-679
  replay (fig 3)                                                                                                                                                                                                                     [2] Louie, K; Wilson, MA (2001), Neuron 145-156
                                                                                                                                                                                                                                     [3] Lee, A.K. and Wilson, M.A. (2002) Neuron 36(6):1183-1194
   • slow during                                                   Reproduced
                                                                                                                                                                                                                                     [4] Hinton, G. E., Osindero, S. and Teh, Y. (2006), Neural
     REM sleep                                                     with
                                                                                                                                                                                                                                        Computation 18(7):1527-1554.
   • fast during                                                   permission
                                                                                                                                                                                                                                     [5] Sutskever, I. and Hinton, G. E. (2006) Technical Report UTML
     SWS, sharp                                                    from Fig 2
                                                                                                                                                                                                                                        TR 2006-003.
                                                                   Lee &
     waves                                                                                                                                                                                                                           [6] Hasselmo, M.E. et al (2002), Neural Computation, 14(4): 793-
                                                                                                                                                                                                                                     [7] Kali, S. and Dayan, P. (2004) Nature Neuroscience 7(3):286-
  dynamics                 Figure 3. Top: Smoothed raster plots of                                                                                                                                                                   [8] Chrobak,J.J., Lorincz,A. & Buszaki,G. (2000) Hippocampus
• LTP in phase w           spikes vs sequentially traversed spatial                                                                                                                                                                     10(4):457-65
                                                                                                                                                                              Figure 5: Examples of boundary reconstructions
  theta rhythm             locations (5-second time scale).                                                                                                                                                                          [9] Chrobak & Buszaki, (1994) J Neurosci 14:6160-6170
                                                                                                                                                                              from BVC input patterns (left) and from model’s
• LTD in anti-             Bottom: Same sequence of 6 place                                                                                                                                                                          [10] Hinton, G.E. (2002) Neural Computation, 14(8):1711–1800
                                                                                           Figure 4. Simplified HC architecture used here.                                    reconstruction of input (right) averaged over 5        [11] Hartley, T., Burgess, N., Lever, C., Cacucci, F. and O'Keefe,
  phase w theta            cells fires during slow wave sleep                                                                                                                 binary stochastic samples                                 J. (2000), Hippocampus 10(4):369-379
  rhythm                   (150msec time scale).
                                                                                 This work was supported by funding from the Natural Sciences and Engineering Research Council of Canada (SB, GEH) and CIAR (GEH)

To top