Cosyne 2008 Workshops
March 3-4, 2008
Snow Bird, Utah
Eugene Izhikevich, The Neurosciences Institute, San Diego, CA
Solving the Distal Reward Problem through the linkage of Dopamine Signaling and STDP
Talk Abstract
In Pavlovian and instrumental conditioning, rewards typically come seconds after reward-triggering actions, creating an explanatory conundrum known as “distal reward problem”: How does the brain know what firing patterns of what neurons are responsible for the reward if (1) the patterns are no longer there when the reward arrives and (2) most neurons and synapses are active during the waiting period to the reward? Here we show how the conundrum is resolved by a model network of cortical spiking neurons with spike-timing-dependent plasticity (STDP) modulated by dopamine (DA). Although STDP is triggered by nearly-coincident firing patterns on a millisecond time scale, slow kinetics of subsequent synaptic plasticity is sensitive to changes in the extracellular DA concentration during the critical period of a few seconds. Random firings during the waiting period to the reward do not affect STDP, and hence make the network insensitive to the ongoing activity --- the key feature that distinguishes our approach from previous theoretical studies, which implicitly assume that the network be quiet during the waiting period or that the patterns be preserved until the reward arrives. This study emphasizes the importance of precise firing patterns in brain dynamics and suggests how a global diffusive reinforcement signal in the form of DA can selectively influence the right synapses at the right time.
This talk is based on the paper Izhikevich E. M. (2007) Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling. Cerebral Cortex 17:2443--2452 available at the author's webpage
