‘Noise’ in the Brain Encodes Surprisingly Important Signals
New research shows that perception can be deeply informed by movement: The visual system may constantly process information about even the smallest gestures.
Lenka Šimečková for Quanta Magazine
Introduction
At every moment, neurons whisper, shout, sputter and sing, filling the brain with a dizzying cacophony of voices. Yet many of those voices don’t seem to be saying anything meaningful at all. They register as habitual echoes of noise, not signal; as static, not discourse.
Since scientists first became capable of recording from single neurons 60 years ago, they’ve known that brain activity is highly variable, even when there’s no obvious reason it should be. An animal’s neural response to a repeated stimulus changes considerably from trial to trial, fluctuating in a way that seems almost random. Even in the total absence of a stimulus, “you would just record spontaneous activity, and you would see that it seemed to have a mind of its own,” according to David McCormick, a neuroscientist at the University of Oregon.
“This gave rise to a view of the brain as being somehow either very [noisy], or using some type of high-level statistics to get over this noisiness,” he said.
But over the past decade, that view has changed. It’s become apparent that this purported randomness and variability relates not just to messiness in the brain’s neural mechanics, but also to behavioral states like arousal and stress — states that seem to affect perception and decision-making as well. There’s more to all the noise, scientists realized, than they had assumed.
Now, by analyzing both the neural activity and the behavior of mice in unprecedented detail, researchers have revealed a surprising explanation for much of that variability: Throughout the brain, even in low-level sensory areas like the visual cortex, neurons encode information about far more than their immediately relevant task. They also babble about whatever other behaviors the animal happens to be engaging in, even trivial ones — the twitch of a whisker, the flick of a hind leg.
Those simple gestures aren’t just present in the neural activity. They dominate it.
The findings are changing how scientists interpret brain activity, and how they design experiments to study it.
An Elegant but Outdated Story
Until about a decade ago, most neuroscientists used anesthetized animals in their experiments. This practice enabled them to make incredible strides toward a better understanding of the brain, but it also “led to a highly distorted view of neuronal processing,” said David Kleinfeld, a neurophysicist at the University of California, San Diego. It was a particular drawback in vision research, because the levels of anesthesia involved were often high enough to undercut confidence that the groggy (or unconscious) animals were subjectively seeing anything at all. At the very least, the anesthesia stripped away any real framework or context that might influence the visual process.
As a result, a certain picture of how vision works emerged, one in which signals from the eyes moved through passive sets of neurological filters that created increasingly specialized and complex representations of the environment. Only later in the process did that visual representation get integrated with information from other senses and other brain areas. “It’s tempting to see primary sensory areas as cameras that give an unadulterated view of what’s happening in the world,” said Anne Churchland, a neuroscientist at Cold Spring Harbor Laboratory in New York — but however elegant this model of vision might be, a mountain of evidence has proved it to be far too simplistic.
The brain’s immense interconnectivity fills it with feedback loops that let higher cortical areas talk to lower ones. Over decades of study, researchers have gradually found each region of the brain to be less specialized than labels might suggest: The visual cortex of people who are blind or visually impaired, for instance, can process auditory and tactile information. The somatosensory cortex, and not the motor cortex, was recently found to play a significant role in the learning of motor-based skills. And broader forces like attention, expectation or motivation can affect how people perceive.
Even in the dark, the neurons of the visual cortex continue to chatter.
But things got even stranger in 2010. New experimental methods, developed just a few years earlier, made it possible to record from the neurons of mice running on a treadmill or ball. The neuroscientists Michael Stryker and Cris Niell, then both at the University of California, San Francisco, initially decided to use those techniques to compare the visual activity in sleeping, anesthetized mice with that of mice which were either running or stationary. (Niell has since moved to the University of Oregon.)
Cris Niell, a neuroscientist at the University of Oregon, uses a home-built imaging system and spherical treadmill to measure the neural activity of mice as they run.
Courtesy of University of Oregon
They quickly realized, though, that there was a more interesting comparison to make. “We went into it wanting to study what’s different in awake versus anesthetized,” Niell said, “but what was surprising to us was how different various aspects of the awake state were when the animal was moving versus not.” They found that when a mouse ran, its responses to visual stimuli got larger: The neurons’ firing rates doubled — which was astonishing because in prior research, the firing rates hadn’t gone up that much even in humans and monkeys required to complete tasks that demanded intense visual attention. These effects tapered off as soon as the mouse stopped running.
“That was pretty surprising, because people had thought of primary visual cortex as being a purely sensory area,” Churchland said.
At the time, in keeping with conclusions drawn from previous work on attention and motivation, Stryker and Niell thought that the result might reflect some general behavioral shift in the mice, a transition from an inactive mode to an active one as the animals started to move. Studies in other labs over the next few years confirmed that general arousal alters neural responses considerably. In 2015, for instance, McCormick and his colleagues reported that a mouse’s engagement with a task predicted how well it would perform. That same year, the neuroscientist Jessica Cardin and her team at Yale University began to disentangle the effects of both running and arousal on activity in the visual cortex.
But all these experiments were limited both in how many neurons they could record from at once and in how many behavioral variables they could account for. Experiments that investigated changes in brain activity based exclusively on whether the mouse was running or still, or whether its pupils were dilated as a proxy for arousal, could explain only bits and pieces of neural variability. A more complete explanation was still wanting.
Now, by taking a more global approach to both animal behavior and brain activity, a handful of research groups have provided exactly that.
Finding Sense in the Swirling Kaleidoscope
Kenneth Harris and Matteo Carandini, neuroscientists at University College London, started with a different goal: to characterize the structure of the spontaneous activity in the visual cortex that occurs even when the rodent gets no visual stimulation. They and other members of their joint team at the university’s Cortexlab recorded from 10,000 neurons at once in mice that were free to act as they wanted — to run, sniff, groom themselves, glance around, move their whiskers, flatten their ears and so on — in the dark.
The researchers found that even though the animals couldn’t see anything, the activity in their visual cortex was both extensive and shockingly multidimensional, meaning that it was encoding a great deal of information. Not only were the neurons chatting, but “there were many conversations going on at the same time,” wrote Marius Pachitariu, a neuroscientist at the Howard Hughes Medical Institute’s Janelia Research Campus in Virginia, and a former postdoctoral researcher in the Cortexlab.
At first, the scientists weren’t sure what to make of it. So they tried to explain the “conversations” by relating the brain activity to exactly what the mice were doing at each moment. They took videos of the face of each mouse, analyzing its motion frame by frame not just for single facets of behavior like running speed or pupil diameter, but for anything at all that might explain the neural variability, down to the tiniest tics and twitches.
Those minor behaviors turned out to account for at least one-third of the ongoing activity in the mice’s visual cortex — activity that had previously all been chalked up to mere noise. It was roughly comparable to the activity that an actual visual input would typically cause. “We have this part of the brain called the visual cortex, and you would think that what it does is vision,” Harris said. “And it does do that. But at least as much of its activity has nothing to do with vision.”
“If we look at the mouse as a whole,” McCormick said, “all of a sudden, that general activity, that swirling kaleidoscope of activity in the brain, starts to make sense.” (He and his lab reported similar findings in a recent preprint.) The activity didn’t just reflect the general state of the mouse’s alertness or arousal, or the fact that the animal was moving. The visual cortex knew exactly what the animal was doing, down to the details of its individual movements.
When researchers studied the spontaneous activity of more than 10,000 neurons in the visual cortex of mice, they were surprised to find it to be rich with information about the animals’ seemingly irrelevant movements. In these images from the experiment, neurons flash when they send signals. Each panel monitors a different depth of tissue in the cortex.
Courtesy of Marius Pachitariu and Carsen Stringer
In fact, this wasn’t unique to the visual cortex. “Everywhere in the brain, it’s the same story. The movement signals are just really unmistakable,” said Matt Smear, a systems neuroscientist at the University of Oregon who did not participate in the study. It cements the idea that “certain intuitive notions about the brain are probably wrong.”
Even more striking, the same neurons that encoded sensory or other functional information were the ones explicitly encoding these motor signals. “All of a sudden we’re saying, ‘Wait — maybe the brain isn’t noisy. Maybe it’s actually much more precise than we thought,’” McCormick said.
The Cortexlab’s findings, which were published in Science in April, demonstrated that neuroscientists need to rethink how they interpret animals’ neural responses. (Niell pointed out that a significant amount of variation observed in human functional MRI studies can also be explained by random fidgets, rather than noise or anything related to the task under investigation.) “For instance, every time the mouse was running, we saw this signal in the neurons right before the mouse would start,” said Carsen Stringer, a postdoctoral researcher at Janelia who did her doctoral work in the Cortexlab. “And we thought, ‘Maybe that’s just the mouse thinking to run.’ But really, it’s the mouse whisking [rhythmically moving its whiskers back and forth] right before it’s running.”
But then what are these signals doing, and why do they matter?
Perception as Action
A system in which each neuron channels information about multiple activities at once might seem unworkably convoluted, but the Cortexlab team found that the brain can cope with all that data more easily than we might think. Their analysis revealed that when a stimulus is shown, the incoming information simply gets added on top of the movement-related signals that were already present. In a single neuron, those signals appear jumbled together, impossible to tell apart. But different neurons might convey the same stimulus but different background behaviors, so that if enough neurons are recorded together, it becomes possible to tease vision and movement apart.
The movement signals therefore aren’t hurting the animal’s ability to process sensory information about the outside world. But scientists still need to explore exactly how those signals might help the brain work better. At its core, this discovery reflects the fact that fundamentally, the brain evolved for action — that animals have brains to let them move around, and that “perception isn’t just the external input,” Stringer said. “It’s modulated at least to some extent by what you’re doing at any given time.”
Sensory information represents only a small part of what’s needed to truly perceive the environment. “You need to take into account movement, your body relative to the world, in order to figure out what’s actually out there,” Niell said.
“We used to think that the brain analyzed all these things separately and then somehow bound them together,” McCormick said. “Well, we’re starting to learn that the brain does that mixing of multisensory and movement binding [earlier] than we previously imagined.”
It’s necessary to know how the body is moving to contextualize and interpret incoming sensory information. If you’re running, the visual world flies by, and the visual cortex needs to know that this is driven by your movement. If you’re circling around a monument, the visual cortex needs to know that you didn’t see 20 different statues, but the same statue from 20 different angles. “Where is the stability among this storm of variance?” McCormick said. “That’s why I think that this recent work is very interesting and very important, because we’re starting to see where the stability is.”
“Our brains aren’t just thinking in our heads. Our brains are interacting with our bodies and the way that we move through the world,” Niell said. “You think, ‘Oh, I’m just thinking,’ or ‘I’m just seeing.’ You don’t think about the fact that your body is playing a role in that.” So it makes sense that a mouse might need to integrate movement signals early on (though exactly how, say, the movement of a whisker helps with vision remains unclear).
In fact, it may go beyond that. The integration might help facilitate what’s known as active sensing — whereby an animal actively coordinates its movement with the information it wants to sense and find. Smear is currently studying this in olfaction: He and his colleagues have found that mice synchronize many of their movements to their sniffing rhythm (their primary means of receiving odor information) with surprising precision.
Even more intriguing, such coordination might help with learning.
A Greater Purpose
Harris, Stringer and their colleagues posit that this integration of sensory and motor information creates a mental scaffolding within which reinforcement learning can happen: If a particular action and stimulus in combination correlates with a noteworthy outcome — say, receiving a reward, or finding oneself in danger — this kind of dual neural coding could help the animal predict that outcome next time and act accordingly.
Churchland suggests that movement signals might help the animal learn in even more concrete ways. In September, Churchland, Simon Musall, a postdoctoral fellow in her laboratory, and their colleagues published the results of an experiment in which they monitored brain activity in mice that were performing a task: The animals had to grab a handle to start a trial, and lick one way or another to report a decision. Even though they were focused on their goal, their neural activity continued to erupt into a chorus of voices dedicated to trivial movements seemingly unrelated to the task at hand. “Most of the activity we found in the brain had nothing to do with the decision,” Churchland said. “It was reflecting the movements the animal was making.”
According to Niell, who was not involved in the study, “what was really striking was that even when an animal was doing that kind of a task, which we think of as purely vision, all of these [unrelated] movements … come in to dominate the signal.”
Churchland and her team also found that as each mouse was trained, its movements locked in more on the task. At first, for instance, the mouse would move its whiskers randomly, but as it learned, it would whisk at specific times — when the stimulus was presented, and when the reward was delivered — even though the act of whisking itself had nothing to do with the reward or the training involved in the task.
Churchland speculates that animals may use these types of signals to help them make decisions — that “maybe for them, this is part of the decision-making process,” she said. “Maybe for animals, as for humans, part of what it means to think and make decisions is to move.” She likened it to the ritual a baseball player might perform before stepping up to bat. “It makes me wonder if these fidgets … serve a greater purpose.”
“People tend to think of movements as being separate from cognition — as interfering with cognition, even,” Churchland said. “We think that, given this work, it might be time to consider an alternative point of view, that at least for some subjects, movement is really a part of the cognition.”
Granted, at this point, “some subjects” mostly means rodents. Scientists are now conducting other experiments to test whether this kind of integration happens as pervasively in primates (including humans) and in a similar way.
Nevertheless, researchers agree that the work heralds a shift in how they conduct their experiments on perception — namely, it demonstrates that they need to start paying more attention to behavior, too.
A Surrender of Control
Until now, neuroscientists have taken a more reductionist route. Much of what we know about neural activity came first from recordings of anesthetized animals, and later from animals moving about in a constricted way. The experiments themselves have also been limited. Niell describes it as the “eye exam” model. “When you go to the optometrist, you sit there and say: horizontal, vertical, better, worse, E, A, T,” he said. But that kind of abstract exercise may not be representative of what we typically do in life. “Our brains did not evolve for us to just sit there and passively watch something without doing anything about it.”
Even the new work by Harris’ and Churchland’s teams involved keeping the mouse’s head still to enable readings from the brain. “If the brain is dominated by movement signals when the animal can’t move [its head], then what’s it going to look like when the animal can move?” Smear said.
Scientists are now advocating for additional approaches that get closer to studying animals performing a natural behavior, one that’s intuitive without training. Of course, other challenges come with that: It’s more difficult to determine cause and effect in less controlled settings, for instance.
Even so, Niell has begun studying mice that use their eyesight to catch and eat crickets. “It’s something that the mouse’s brain is wired up to do,” he said, “and also a task where they are moving, and so therefore they have to integrate their movement with what’s out there.” He and his colleagues have now found that certain types of already-discovered neurons serve precise behavioral roles in the capture of prey.
“What we think of as being weird or unusual signals,” Niell said, “might start to make sense when you actually let an animal do what it would normally do, and not train the mice to be like little humans.”
McCormick agreed. “We had a very impoverished view of the brain,” he said. “I wouldn’t say we have a perfect view now, but … we have a richer view that needs to grow.”
Editor’s note (added Nov. 8, 2019): The described work conducted in the Cortexlab and by Churchland’s group received funding from the Simons Foundation, which also funds this editorially independent magazine.
