Cosyne 2008 Workshops
March 3-4, 2008
Snow Bird, Utah
Speaker Name
Wulfram Gerstner, Laboratory of Computational Neuroscience, LCN EPFL, Lausanne, Switzerland
Talk Title
Spike-based reinforcement learning of navigation
Talk Abstract
We have studied a spiking reinforcement learning model derived from reward maximization (Pfister et al., 2006; Florian 2007) where causal relations between pre-and postsynaptic activity set a synaptic eligibility trace (see also Florian 2007, Izhikevich 2007). The synapse is updated when a global reward signal (such as dopamine) is received. We have used the learning algorithm in a model of the Morris Water Maze task. Reward is given only if the simulated rat finds the hidden platform. The simulated rat explores the environment in random search. After only a few trials (about 10), the rat has learned to approach the goal from arbitrary start conditions. The model features automatic generalisation in state and action space due to coding by overlapping profiles of place cell and action cells.
joint work with E. Vasilaki, R. Urbancik, and W. Senn
References:
J.-P. Pfister et al., Neural Computation 18:1309-1339 (2006)
R.V. Florian, Neural Computation 19:1468-1502 (2007)
E. Izhikevich, Cerbral Cortex (2007)
