Imagine yourself hurtling through space with millions of stars passing by. How is your brain able to perceive the direction your ship is moving toward? Li Li, Professor of Neural Science and Psychology, explores this question in her latest paper, “Integration of Motion and Form Cues for the Perception of Self-Motion in the Human Brain,” published in the field’s leading journal, The Journal of Neuroscience. Her team, along with the team of her collaborator, Dr. Kuai Shuguang of East China Normal University, have uncovered the areas of the human brain that can integrate form and motion cues for the perception of the direction we are traveling.
A simulation of hurtling through space.
“Accurate perception and control of our self-movement in the world is essential for human survival,” Li said. “When moving at a high speed, rapidly passing objects will form radially expanding motion patterns in our eyes. The information contained in these motion patterns is essential for us to perceive and control our self-movement. Understanding such information and the related neural mechanism can help the design of biomimetic robots and the treatment of neurodegenerative motor disorders.”
A dominant theory in neuroscience proposes that motion cues (dynamic visual information related to movement) and form cues (static visual information obtained from the scenery around us) are processed along separate visual pathways in the brain. But several studies have challenged this theory. For instance, in an earlier study, Li and her team found that people use both form and motion cues to determine the direction of travel. Building on these findings, Li and her team, including research assistant Shan Zhoukuidong and postdoctoral fellow Chen Jing, sought to pinpoint the brain areas where the integration takes place.
This figure shows how the center or focus of expansion (FoE) for motion cues (marked with a red +) differs from the FoE for form cues (marked with a blue x) in the same frame of a video. See a sample clip of video here.
In this study, Li’s team developed short video displays that contained pairs of dots moving in a radial pattern to simulate what people see when they are moving forward. Each video contained two visible centers. The motion-defined focus of expansion (FoE) is the center where the dot pairs move outward in a radial pattern. The form-defined FoE is the center where all the dot pairs were oriented toward. The two centers could be the same or different from each other.
Using functional magnetic resonance imaging technique, Li’s team took and analyzed high resolution pictures of participants’ brains as they watched the short video displays, identifying those brain areas that responded to the motion and forms cues in the displays.
By fixing the motion-defined FoE at the center of the display and shifting the location of the form-defined FoE from left to right, Li’s team showed that both early (V1 and V2 shown in the picture below) and high-level visual areas of the brain (V3v, V3d, V3a, and V3B/KO shown in the picture below) appeared to be responding to the shifts in form-defined FoE, indicating that these areas process form cues.
Then, they shifted the locations of both the motion-defined FoE and the form-defined FoE so that the centers were either overlapping or non-overlapping (see the pictures below). The overlapping displays led to distinct percepts of the direction of travel while the non-overlapping displays led to the same percept. They found that area V3B/KO responded according to percepts but not physical shifts of the two centers, suggesting that it is the brain area that integrates motion and form cues to form a unified percept of the direction of travel. After that, Li’s team randomized the motion or the form cues in the display and confirmed that V3B/KO responded to global but not local information.
The findings of the current study have many practical applications. Understanding the neural process for how humans integrate different visual cues to perceive their direction of travel can help the design of self-driving vehicles and biomimetic robots. The findings can also be applied to the treatment of neurodegenerative motor disorders. “If lesions occur in certain areas of a patient’s brain, such as V3B/KO, we would be able to predict how the disease could affect the patient’s perception and control of self-movement in the world,” Li said.
Though Li’s team is proud of their work, finding area V3B/KO is not the end of the journey. The lab will continue to investigate effective visual and non-visual cues and strategies people use for walking and driving, and the related underlying neural mechanisms. “This area still has a lot of questions that remain unanswered,” Li said, “and we hope to make a significant contribution to the advancement of this field in the coming years.”
Li is the primary corresponding author of this paper. In addition to Li’s team at NYU Shanghai, Dr. Kuai Shuguang, graduate students Xu Zhexin and Li Jiamei at East China Normal University also contributed to this research. This research was funded primarily by the NYU-ECNU Institute of Brain and Cognitive Science at NYU Shanghai.