Cosyne 2008 Workshops
March 3-4, 2008
Snow Bird, Utah
Speaker Name
Michael Hasselmo, Center for Memory and Brain Department of Psychology and Program in Neuroscience, Boston University, Boston, MA
Talk Title
Cortical mechanisms for memory-guided behavior: Oscillations, grid cells, arc length and RL
Talk Abstract
Reinforcement learning theory (Sutton and Barto, 1998) provides a useful framework for modeling goal-directed action selection using look-up tables, but does not specify neural mechanisms for action values and action selection. For several years, I have modeled neural mechanisms for goal-directed action selection in behavioral tasks. These models use active maintenance of a goal representation with cyclical phases of backward spread of activity from this goal location that converge with forward spread of activity from a representation of current state to activate a representation of the next action appropriate for approaching the goal. This process was used in models of goal-directed action selection in prefrontal cortex using threshold units (Hasselmo, 2005) and integrate-and-fire neurons (Koene and Hasselmo, 2005) and compared with the performance of the SARSA algorithm. An earlier model used a similar process in modeling an interaction in hippocampal region CA1 between backward spread in entorhinal cortex and forward spread in region CA3 (Hasselmo et al., 2002).
We have also explored mechanisms by which the memory-dependent decision processes in rat behavioral tasks can be transformed into MDPs for reinforcement learning theory. In a simulation of physiological and behavioral data on spatial alternation, I used theta rhythm oscillatory dynamics for sequence retrieval in hippocampus that interact with reinforcement learning mechanisms to allow memory-guided action selection (Hasselmo and Eichenbaum, 2005). This replicated properties of context-dependent firing and theta phase precession of hippocampal region CA1 neurons. That model used memory retrieval at every state, but in a recent model we implemented memory actions that trigger selective encoding and retrieval of states only when necessary for performance of behavioral tasks (Zilli and Hasselmo, 2008).
Recent work in my laboratory demonstrated potential oscillatory mechanisms underlying the neural representation of state by grid cells in entorhinal cortex (Giocomo et al., 2007; Hasselmo et al., 2007). We showed that entorhinal stellate cells have systematic changes in frequency of membrane potential oscillations that correlate with the systematic changes in the size and spacing of firing fields along the dorsal to ventral axis of entorhinal cortex (Giocomo et al., 2007). Using a model by Burgess, Barry and O’Keefe, we showed how interference of membrane potential oscillations can integrate spatial state in the form of oscillatory phase and directly underlie the periodicity of grid cell firing (Hasselmo et al., 2007). This inspired further models of oscillatory mechanisms for coding of arc length (Hasselmo, 2007). These models allow simulation of context-dependent firing of hippocampal neurons that could contribute to memory-guided action selection.
Giocomo LM, Zilli EA, Fransen E, Hasselmo ME. (2007) Temporal frequency of subthreshold oscillations scales with entorhinal grid cell field spacing. Science. 315(5819):1719-22.
Hasselmo, M.E. (2005) A model of prefrontal cortical mechanisms for goal directed behavior. Journal of Cognitive Neuroscience, 17(7):1115-29.
Hasselmo ME (2007) Arc length coding by interference of theta frequency oscillations may underlie context-dependent hippocampal unit data and episodic memory function. Learning and Memory, 14: 782-794.
Hasselmo, M.E. and Eichenbaum, H. (2005) Hippocampal mechanisms for the context-dependent retrieval of episodes. Neural Networks, 18(9):1172-1190.
Hasselmo ME, Giocomo LM, Zilli EA (2007) Grid cell firing may arise from interference of theta frequency membrane potential oscillations in single neurons. Hippocampus, 17(12): 1252-1271.
Hasselmo, M.E., Hay, J., Ilyn, M. and Gorchetchnikov, A. (2002) Neuromodulation, theta rhythm and rat spatial navigation. Neural Networks, 15: 689-707.
Koene, R.A. and Hasselmo, M.E. (2005) An integrate and fire model of prefrontal cortex neuronal activity during performance of goal-directed decision making. Cerebral Cortex, 15(12):1964-81.
Zilli EA, Hasselmo ME. (2008) Modeling the role of working memory and episodic memory in behavioral tasks. Hippocampus, 18(2):193-209
Article PDF files and sample MATLAB scripts are available at www.bu.edu/hasselmo/publications.html.
