How Home Alone Helped UMD Neuroscientists Unlock Brain Scan Data

The vast majority of analyses of brain scans recorded via functional magnetic resonance imaging (fMRI) are static. They can show the spatial distribution of brain signals at a given moment, but they do not shed light on how the brain responds over longer periods of time. Yet this temporal information is of vital importance. Seeing how the human brain represents subtle signal fluctuations across time is a crucial step in understanding the activity of the brain in contexts that more closely approximate daily life.

Now University of Maryland neuroscientists and computer scientists have teamed up to develop a new method to unearth this important temporal information.

In a paper published last month in PLOS Computational Biology, Luiz Pessoa, professor of psychology and director of the Maryland Neuroimaging Center, and Joseph JaJa, professor and interim chair of electrical and computer engineering with a joint appointment at the University of Maryland Institute for Advanced Computer Studies, engineer a novel technique for training a series of algorithms to disclose temporal information otherwise hidden in fMRI brain scans.

The neural network algorithms, together with novel statistical approaches, were applied to a large public data set of fMRI scans of subjects watching clips of humorous or suspenseful films like Home Alone, Star Wars, and Ocean’s 11.

The paper and collaboration were borne of a 2017 seed grant award from the Brain and Behavior Institute (BBI).

“Neuroscience is incorporating extremely sophisticated data analysis, including machine learning, neural networks, and advanced statistical methods,” said Pessoa. “The collaboration with Joseph JaJa allows neuroscientists to work together with engineers to advance our understanding of the brain. We were very fortunate for the support from the BBI, which allowed us to get the project started, and from the Department of Electrical and Computer Engineering, which granted me a courtesy appointment allowing me to advise students with tremendous potential to tackle extremely complex problems.”

The researchers’ hypothesis was that, once a neural network has digested the data in these brain scans, it would recognize similar information in new fMRI scans—information that would otherwise remain latent. Whereas analyses of fMRI brain scans typically only capture a “snapshot” of the brain—that is, a subject’s reaction to a single, known stimulus—Pessoa and JaJa show how so-called recurrent neural networks can extract information about the brain’s responses over longer timescales.

“The BBI is proud to support exciting new research partnerships across the life and physical sciences such as the collaboration between Pessoa and JaJa,” said Elizabeth Quinlan, professor of biology, Clark Leadership Chair in Neuroscience, and director of the BBI. “The development and application of machine learning techniques and artificial intelligence approaches are driving the vital growth and expansion of collaborative neuroscience on our campus.”

With recent funding to Pessoa from the National Institute of Mental Health, the researchers plan to continue studies into the brain mechanisms of emotion and motivation.

“We need to break down silos and promote integration across disciplines,” Pessoa noted, emphasizing the advantages of interdisciplinary science. “This cross-disciplinarity is the only way that we can make serious progress when trying to understand one of the most challenging problems in all of science—the brain.”

###

Media Relations Contact: Nathaniel Underland, underlan@umd.edu

About the Brain and Behavior Institute: The mission of the BBI is to maximize existing strengths in neuroscience research, education and training at the University of Maryland and to elevate campus neuroscience through innovative, multidisciplinary approaches that expand our research portfolio, develop novel tools and approaches and advance the translation of basic science. A centralized community of neuroscientists, engineers, computer scientists, mathematicians, physical scientists, cognitive scientists and humanities scholars, the BBI looks to solve some of the most pressing problems related to nervous system function and disease.

Published November 1, 2021