The Brain Reshapes Our Malleable Senses to Fit the World

The eye is something like a camera, but there is a whole lot more to vision than that. One profound difference is that our vision, like the rest of our senses, is malleable and modifiable by experience. Take the commonplace observation that people deprived of one sense may have a compensatory increase in others — for example, that blind people have heightened senses of hearing and touch. A skeptic could say that this was just a matter of attention, concentration and practice at the task, rather than a true sensory improvement. Indeed, experiments show that a person’s sensory acuity can achieve major improvement with practice.

Yet with modern methodologies, neuroscientists have conclusively proved that the circuits of the brain neurons do physically change. Our senses are malleable because the sensory centers of the brain rewire themselves to strike a useful balance between the capacities of the available neural resources and the demands put on them by incoming sensory impressions. Studies of this phenomenon are revealing that some sensory areas have innate tendencies toward certain functions, but they show just as powerfully the plasticity of the developing brain.

Take a rat that has been deprived of vision since birth — let’s say because of damage to both retinas. When the rat grows up, you train that rat to run a maze. Then you damage the visual cortex slightly. You ask the rat to run the maze again and compare its time before the operation and after. In principle, damaging the visual cortex should not do anything to the maze-running ability of that blind rat. But the classic experimental finding made decades ago by Karl Lashley of Yerkes Laboratories of Primate Biology and others is that the rat’s performance gets worse, suggesting that the visual cortex in the blind rat was contributing something, although we do not know what it was.­­

During the same era, clinicians working on human patients reported two kinds of developmentally induced blindness. In the first, a patient who from birth had one eye occluded — from a cataract, for example, or from rare eyelid problems — but then had that anatomical problem removed still ended up with one blind or nearly blind eye. Something about the early occlusion kept that eye and its central neural pathways from hooking up properly.

The second type of developmentally induced blindness concerned children who were born cross-eyed, with their eyes pointing in different directions. When the children grew up, it was all too often found that one eye or the other had taken over: one eye worked, and the other one did not. This is called colloquially “lazy eye”; the technical term is amblyopia. The eye is not truly blind — you can show that the retina is working — but the person has no useful vision through it. (There are various therapies for this condition at present, the commonest of which is to patch alternating eyes during early childhood so that one eye never has a chance to take over and suppress the other.)

Damaged Senses Rewire Themselves

The vision pioneers David Hubel and Torsten Wiesel, who discovered image processing in the visual cortex, repeated these experiments in animals and discovered the neural basis of the lazy eye. During a critical period in early life, the synapses that connect the retinal output to the central nervous system are malleable. If cortical neurons get a lot of conversation from one eye and none from the other eye, axons representing the first eye grab all the synaptic spaces on the cortical neurons. This leaves the second eye functional but with no cortical neurons to talk to.

For crossed eyes, the scientists found, it’s a bit more subtle. In normal circumstances, images from one eye and images from the other eye are almost perfectly in register, and the same spot in the visual scene stimulates a single group of cortical neurons. When Hubel and Wiesel artificially crossed the eyes of animals, however, by making a young animal wear a prism that shifted its visual image, images from its two eyes did not properly converge on the same brain target. The person sees double, literally: two separate and conflicting images. The brain has to choose one eye or the other. Connections from one eye are suppressed — first temporarily but after a while permanently, leaving that eye functionally blind.

A clever experiment demonstrates a different kind of reorganization of cortical responses. Under normal circumstances, there is a “map” of the retina on the visual cortex. To be sure, it is distorted by the undulations of the cortex’s surface, but you can see very directly that neighboring points on the retina project to neighboring points on the visual cortex, creating an organized map of the visual scene on it. The experiment was to painlessly make a very small hole in the retina of a monkey using a laser. The experimenter, Charles Gilbert of Rockefeller University, then recorded from the visual cortex to see how the cortical map had responded. Initially, there was a hole in the cortical map of the visual space, corresponding to the hole in the retina. After a while, though, neighboring regions of cortex moved over to occupy the vacated cortical space: Neighboring regions of the retina communicated with the cortical cells that normally would have responded to the damaged region.

This does not mean that vision was restored for the damaged region of retina. If you have a lesion in your retina, you are never going to see anything in the region that was destroyed — you have a blind spot. But even though the brain can never compensate for the hole in the retina, the region around the retinal lesion will “own” more cortical neurons than it did previously.

One way to think of this is as nature’s way to prevent cortical idleness. If an area of cortex is no longer receiving inputs from its natural place, it would be wasteful for that area of cortex to be forever inactive. Instead, after a while, its function is given over to undamaged inputs. In the more general case, you can easily imagine this mechanism as a way of dealing with small strokes. (Neuropathologists tell us that we all incur these small losses of brain tissue during the course of our lives.) Imagine that you have a tiny cortical stroke, affecting only a very small blood vessel, and that the region of brain it feeds dies. It would be wasteful of precious cortical resources for areas of the brain that used to receive input from the region that is now damaged by the stroke to be forever silent. Instead, the brain makes the best of a bad situation by giving those brain areas over to their neighbors.

Reorganizing Normal Perception

The senses adapt to various types of neural damage, which are pretty crude events on the big scale of neural life. But there are also subtler reorganizations that occur naturally and happen to all of us.

One of the striking indications of brain plasticity came from scanning the brain activity of people who had been blind from birth. When blind volunteers used their fingers to read Braille while in the scanner, the brain areas usually occupied by processing visual input — again, the primary visual cortex — were activated. Somehow, the processing of tactile information had taken over the unused visual center.

Another dramatic example came from a study of violinists. To play the violin, you make large, relatively crude motions with one arm as the bow sweeps up and down across the strings. With the other hand you make a series of very subtle movements, depressing the strings at varying, tightly defined locations up and down the violin’s fingerboard — very quickly if you are a good violinist, astonishingly quickly if you’re a star. This is a remarkable task for the speed and precision it requires. Professional violinists practice these movements for hours each day.

This has a consequence on the physical arrangement of the connections in their brains, because movements of the fingers are controlled by a specific brain area. In professional violinists, the area expands, even pushing aside functions from neighboring brain tissue. But this occurs only for the hand that fingers the strings. The same regions on the other side of the brain, which control the other hand, have no expansion because the required movements of that hand are relatively crude.

(Violinists are an extreme example, but I wonder what happens in other cases, too. If you are a professional athlete, do your muscle-control brain circuits expand at the expense of others? If you spend much of your working life worrying about the brain, do the worrying-about-the-brain circuits expand at the expense of the appreciating-opera circuits?)

The opposite situation — deprivation rather than overuse — has been arranged in the laboratory. Cats raised in darkness lost the ability to properly fuse images from their two eyes. Other cats were raised under conditions where the only patterned vision provided to them was of vertical or horizontal stripes. Stripe-reared animals grew up with a bias in the orientation selectivity of the neurons in their primary visual cortex: An abnormally high number of cells were tuned to vertical orientations if the cat’s only visual experience had been vertical stripes, horizontal if the cat saw only horizontal stripes.

A clever variation on dark-rearing was to deprive animals during early life of the ability to see motion. The experimenters did this by rearing cats in an environment lit only by very brief strobe flashes. This allowed the cats to see, but the flashes were too short for any meaningful movement of objects across the retina to occur. What happened? These animals grew up without direction-selective neurons in their cortex.

All of these findings and others point to malleability in the organization of the sensory systems. But how important is this under natural human conditions? What happens if a person grows up without any vision?

Learning to See

The neuroscientist Donald Hebb predicted that vision is to a major extent learned. Complex perceptions are formed through experience, by association, because objects in the world occur in clusters of individual features. He believed that this had to happen early in life, before the brain became unable to form the necessary new assemblies. His basic idea was right: Much of vision does depend on visual experience. But his conclusion that this had to happen at a young age seems to be only partly true.

The evidence comes from experiments in which individuals blind from birth were later given sight. Pawan Sinha of the Massachusetts Institute of Technology realized during a visit home that there were perhaps 300,000 children born with dense congenital cataracts in the villages of India. In these children, the lens of the eye is replaced by a cloudy fibrous tissue. The cataracts allow in light and dark but deprive the child of all detailed vision. In a brilliant combination of humanitarianism and science, Sinha organized a program to search for these children and transport them to New Delhi, where surgeons in a modern hospital replaced their lenses with clear synthetic ones — the same cataract operation carried out for many aging individuals.

Sinha’s team tested his patients’ vision before the operation, immediately after it and months or years later. Taking away the cataract did not immediately restore detailed vision in the children. The world to them seemed a confused blur. But as time passed they began to see, and after a few months they could see details beyond simply light and dark. Many could walk without a white cane, ride a bicycle in a crowded street, recognize friends and family, attend school and carry out the other activities of a sighted person.

Yet their vision seems never to have become perfect. Their visual acuity remained below normal, even after months of training. One patient commented that he could read headlines in the newspaper but not the finest print. Some had trouble with specific visual tasks, such as separating two forms that overlap each other.

So it seems that much vision can be restored, but that the plasticity of the visual system is not limitless. Further evidence of this comes from the behavior of the cortical regions in primates’ inferior temporal lobe termed “face patches” because they respond only to faces as a visual stimulus.

First, the fact that the face patches have reproducible locations in different individual people (or monkeys) shows that the brain has some level of intrinsic pattern for them. Second, as the newly sighted Indian children learned to see, their brain patterns underwent a change. Right after the cataract removal, functional magnetic resonance image scans (fMRIs) showed a disorganized, widespread response to visual input, including faces, but it quickly changed to a series of patches — and the patches were in their normal locations. This shows that the brain knew ahead of time where the face patches were supposed to be; it is evidence for at least a low level of predetermination of the visual structures. The vision researcher Margaret Livingstone calls these predetermined locations “proto-face patches.”

Finally, a powerful and elegant experiment on sensory neural plasticity was published in late 2017 by Livingstone and her colleagues. They raised monkeys from birth in an environment where they never got to see a face. Not a human face, not a monkey face, no faces at all. The monkeys were cared for lovingly, but whenever they were near a monkey, the experimenters wore a welder’s mask.

The monkeys otherwise grew up in a completely normal visual world: They could see everything in their cage and in the surrounding room; they could see the experimenters’ torso, arms and feet; they could see the baby bottle with which they were fed. They could hear the normal sounds of a monkey colony. Their only deprivation was that they never saw faces. These monkeys developed in most ways normally, and when they were introduced into the monkey colony after the experiment was finished, they socialized happily with their peers and integrated successfully into monkey society.

After the experimenters trained these monkeys to lie still inside the fMRI scanner, they tested the monkeys by showing them various things, including faces. As you may have guessed, they grew up without face patches in their brains. Remarkably, though, what would normally have been the temporal lobe’s face recognition areas instead responded to images of hands. In a normal social environment, the most important visual objects for a primate are faces. Faces signal anger, fear, hostility, love and all of the emotional information important to survival and thriving. Apparently, the second most important feature in the environment is hands — the monkeys’ own hands, and the hands of the experimenters who nurtured and fed them.

Although what would normally have been face patches turned into “hand patches,” this preference was still somewhat plastic. About six months after the monkeys were allowed to see the faces of the experimenters and of other monkeys, the cells in the face patches gradually reverted to being face-sensitive. Evidently, faces convey so much important information that they recaptured the brain territory that had been taken over by hands.

The existence of face patches explains a curious and long-recognized clinical observation. There is a condition known as face blindness (prosopagnosia, from the Greek prosop, “face,” and agnosia, “ignorance”) in which a person’s vision is quite normal except for difficulty recognizing faces. The sufferer can see fine, is as good as anyone else at distinguishing one face from another, but has difficulty recognizing faces from memory.

There are gradations of prosopagnosia ranging from almost complete, which may bring the person to medical attention, to very mild. Speaking personally, I am well on the prosopagnostic side. It is an embarrassing problem. I can spend a pleasant evening at dinner with you, and the next day pass you in the hall and think, “Do I know that person?” but can’t get any further than that. So if I have cut you cold at one time or another, please understand that it was my disability speaking, not any lack of interest in you.

From the book WE KNOW IT WHEN WE SEE IT: What the Neurobiology of Vision Tells Us About How We Think by Richard Masland. Copyright © 2020 by Richard Masland. Reprinted by permission of Basic Books, New York, NY. All rights reserved.