Cosyne 2009 Workshops
March 2-3, 2009
Snow Bird, Utah
Workshop Title
Eye movements and visual perception
Organizer(s)
Dario Ringach, UCLA
Abstract
This workshop will address the inextricable link between eye movements and the representation of the visual image in early visual processing, and how eye movements can influence perception and vice-versa.
Topics to be discussed include how the visibility of objects may be (paradoxically) improved during eye movements (both during fixation and tracking), how the natural statistics of the environment may be used to general optimal eye movements for certain tasks (such as visual search), and how eye movements alter our perception of space and time.
How active sensing (of which eye movements is one example) shapes the early representation of sensory information, and how we can incorporate active sensing into models of perception and action, is a key question that is attracting increased attention across different systems.
The participants in the workshop were selected for their unique ability to combine experimental and computational work and the high quality of their work. They have all agreed to showcase their experimental data and computational modeling in the talks for the workshop.
Schedule
Morning session (8:00 - 11:00 am)
8:00 - 8:30 am Michele Rucci (Boston Univ) Visual Encoding with Jittering Eyes
8:40 - 9:10 am Rich Krauzlis (Salk) A Neural Mechanism for Microsaccade Generation in the Primate Superior Colliculus
9:20 - 10:00 am Karl Gegenfurtner (Giessen University) Visual perception during smooth pursuit eye movements
10:10 - 10:40 am Dario Ringach (UC Los Angeles) Dynamics of gaze stabilization in fly and man
10:40 - 11:00 am Open discussion of first half
Afternoon session (4:30 - 7:30 pm)
4:30 - 5:10 pm Marty Banks (UC Berkeley) Corresponding points, binocular eye movements, and the ground plane
5:20 - 5:50 pm Bill Geisler (UT Austin) Ideal Observer Analysis of Fixation Selection in Visual Search
6:00 - 6:30 pm Miguel Eckstein (UC Santa Barbara) Virtual evolution for visual search in natural images results in behavioral receptive fields with inhibitory surrounds
6:40 - 7:20 pm Concetta Morrone (Istituto di Neuroscienze CNR) A model for the perception of space and time during saccade
7:20 - 7:30 pm Open discussion of second half
Speakers
Michele Rucci (Boston Univ), Visual Encoding with Jittering Eyes
During visual fixation, small eye movements continually displace the stimulus on the retina. In this talk, I will summarize a theory for the existence of fixational eye movements, which links the normal instability of visual fixation to the statistics of natural scenes. According to this theory, fixational eye movements contribute to the neural encoding of natural scenes by attenuating input redundancy, emphasizing the elements of the stimulus that cannot be predicted from the statistical properties of natural images, and enabling a temporal multiplexing of visual information from the retina to the cortex. I will summarize experimental results based on predictions from this theory, which show contributions of fixational eye movements to the perception of fine spatial detail. Finally, I will describe recent findings on the way the visual system separates the retinal image motion caused by eye movements from the motion caused by objects in the scene.
Rich Krauzlis (Salk), A Neural Mechanism for Microsaccade Generation in the Primate Superior Colliculus
During fixation, the eyes are not still, but often exhibit microsaccadic movements. The function of microsaccades is controversial, largely because the neural mechanisms responsible for their generation are unknown. In this talk, I will present evidence that the superior colliculus (SC), a retinotopically organized structure involved in voluntary-saccade target selection, plays a causal role in microsaccade generation. Neurons in the foveal portion of the SC increase their activity before and during microsaccades with sizes of only a few minutes of arc, and exhibit selectivity for the direction and amplitude of these movements. Reversible inactivation of these neurons significantly reduces microsaccade rate without otherwise compromising fixation. These results, coupled with computational modeling of SC activity, demonstrate that microsaccades are controlled by the SC, and provide an explanation for the link between microsaccades and visual attention.
Karl Gegenfurtner (Giessen University), with Alexander C. Schütz, Miriam Spering & Doris I. Braun. Visual perception during smooth pursuit eye movements
When we view the world around us, we constantly move our eyes. Saccadic eye movements bring objects of interest into the fovea and smooth pursuit eye movements keep moving objects there. We wanted to explore the functional role of pursuit eye movements in particular. Therefore, we performed a whole variety of visual tasks during fixation and pursuit. Our results show that visual sensitivity during pursuit can even be higher than during fixation for chromatic and high spatial frequency luminance stimuli. Object recognition for pursuit targets is nearly as accurate and nearly as fast as during fixation. While this indicates that pursuit is optimized to serve object recognition, pursuit also has benefits for the perception of visual motion. In a task where subjects had to judge whether a small spot would hit a line target, they were notably better when the target was pursued. In a speed discrimination task where a small speed perturbation was applied to one of several moving targets, it was significantly easier to localize the change during pursuit. These findings indicate that pursuit not only supports the recognition of moving objects by keeping them in foveal vision, but also plays an important role in judging visual motion.
Dario Ringach (UC Los Angeles). Dynamics of gaze stabilization in fly and man.
A common problem faced by moving organisms is how to stabilize gaze during self-generated movement or external forces. Here we studied the the dynamics and linearity of of gaze stabilization in flies and humans using white-noise analysis techniques. In flies, we investigated how perturbations of the optic-flow field pattern (simulating perturbations of flight) generates commands that drive wing kinematics that stabilize gaze. In humans, we examined how perturbations in the velocity of a moving target generates a command for pursuit eye movements. The method relies on the estimation of the optimal linear filter linking perturbations of the visual stimulus to the corresponding motor variables. We found pure delays to be very small in both cases (~60ms), but the integration time in flies was substantially longer. Further, the optimal linear filter is sufficient to predict the behavior to novel stimuli with reasonable accuracy.
Marty Banks (UC Berkeley). Corresponding points, binocular eye movements, and the ground plane.
Corresponding points are loci in the two retinas that when stimulated give rise to the same perceived visual direction. The horopter is the set of points in space that stimulate corresponding points. The horopter has special status functionally because depth discrimination from stereopsis is much finer for stimuli on the horopter than for stimuli off the horopter. Helmholtz observed that corresponding points above and below the foveas are shifted in opposite directions in the two eyes. He realized that the shifts were appropriate for placing the horopter in the ground plane when a standing observer fixates a distant point on the ground. The shift pattern has been called the Helmholtz shear. He speculated that the shear is an evolutionary adaptation to place the region of finest binocular vision in the ground. We examined what happens when a standing observer with the head upright makes eye movements to fixate distant, medium-range, and near points in the ground. Amazingly, the horopter remains on or near the ground plane in all cases. We will show that this is a consequence of binocular eye movements obeying Listing’s Law. We also examined what happens when the observer pitches the head downward to aid fixation in the ground. Again, the horopter remains near the ground plane because the plane of the rotation axes of the eyes rotates slightly relative to the head. Thus, shifts in corresponding points cause the region of finest binocular vision to remain in or near the ground plane as a standing observer looks from near to far. Humans are thereby better able to distinguish deviations from the ground plane (obstacles, potholes, etc.) while moving through the environment.
Bill Geisler (UT Austin). Ideal Observer Analysis of Fixation Selection in Visual Search
The primate visual system combines a wide field of view with a high resolution fovea and uses saccadic eye movements to direct the fovea at potentially relevant locations in visual scenes. This is a sensible design for a visual system with limited neural resources. However, to be effective this design requires sophisticated task-dependent mechanisms for selecting fixation locations. I will argue that in studying the brain mechanisms that control saccadic eye movements in specific tasks, it can be very useful to consider how fixations would be selected by an ideal observer. Such an ideal-observer analysis provides: (i) insight into the information processing demands of the task, (ii) a benchmark against which to evaluate the actual eye movements of the organism, (iii) a starting point for formulating hypotheses about the underlying brain mechanisms, and (iv) a benchmark against which to evaluate the efficiency of hypothesized brain mechanisms. I will also argue that standard visual search tasks have certain characteristics that are unlike most natural visual search and that a new family of search tasks, without these unnatural characteristics, is not only more representative of natural search but easier to model and analyze mathematically. In making these arguments, I will describe recent examples from our lab concerning naturalistic visual-search tasks.
Miguel Eckstein (UC Santa Barbara) Virtual evolution for visual search in natural images results in behavioral receptive fields with inhibitory surrounds
The neural mechanisms driving perception and saccades during search use information about the target but often show an inhibitory surround not present in the target luminance profile (e.g., Eckstein et al, J. Neurosc., 27, 1266-70, 2007). Here, we ask whether these inhibitory surrounds might reflect a strategy that the brain has adapted for the search of targets in natural scenes. To test this hypothesis we sought to estimate the best linear template (behavioral receptive field), built from linear combinations of V1 simple cells (Gabor channels), for search for an additive Gaussian target embedded in natural images. Statistical non-stationarity and non-Gaussian properties of natural scenes precludes calculation of the bet linear template from analytic expressions and requires an iterative method such as genetic algorithms (virtual evolution). Thus, we virtually evolved a behavioral receptive field built from linear combinations of a subset of V1 simple cells. We found that the optimal linear template includes inhibitory surrounds, albeit larger than those found in humans for search in white noise. Our results suggest that the apparent sub-optimality of inhibitory surrounds in human behavioral receptive fields when searching for a target in white noise might reflect a strategy to optimize detection of signals in natural scenes. Finally, we suggest that optimized linear detection of spatially compact signals in natural images joins previous hypotheses (decorrelation of visual input and sparse representations; e.g., Graham et al., Vis. Res., 46, 2901-13, 2006; Atick & Redlich, 1992) as a possible theory for the center-surround organization of receptive fields in early vision.
Concetta Morrone (Istituto di Neuroscienze CNR). A model for the perception of space and time during saccade
Vision is always clear and stable, despite continual saccadic eye movements, ballistic movements of the eyes that reposition our gaze, two to three times a second. The world seems to stay put, while comparable image flow produced externally, rather than by movements of the observer’s own eyes, has an alarming effect on the observer’s sense of stability. The perceptual and neuronal mechanisms mediating stability are still mostly unknown, although much evidence points to the existence of an eye-positional signal that becomes active before the actual eye movement. Before the onset of saccades, briefly presented stimuli are grossly mislocalized in space and in time: visual stimuli appear mislocalized in space, usually in the direction of the saccade, and delayed in time; and both relative distances and durations appear strongly compressed (Ross et al Nature 1997; Morrone et al., Nature Neuroscience 2005; Binda et al. J Neuroscience 2007). The temporal distortions also depend on perceived spatial position: the amount of perceived perisaccadic spatial displacement correlating with the magnitude of temporal distortion, but not with the physical position of the target. These data are consistent with the idea that the localization of visual events in space and in time is mediated by neurones with craniotopic receptive fields that commence to compensate for the gaze shift before the actual eye position. Here we will present a model that implements an early and rapid updating of spatial receptive fields, that simulates the perceptual localization errors in space and time. The update (shift in retinal position of the receptive fields) is mediated by a sluggish noisy eye-position signal, obtained by integrating optimally the output of two populations of neural activity, one centered at the current point of gaze, the other centered at the future point of gaze. The model also simulates the perceptual stability of long-lasting stimuli.
