The Strange Second Life of String Theory

String theory strutted onto the scene some 30 years ago as perfection itself, a promise of elegant simplicity that would solve knotty problems in fundamental physics — including the notoriously intractable mismatch between Einstein’s smoothly warped space-time and the inherently jittery, quantized bits of stuff that made up everything in it.

It seemed, to paraphrase Michael Faraday, much too wonderful not to be true: Simply replace infinitely small particles with tiny (but finite) vibrating loops of string. The vibrations would sing out quarks, electrons, gluons and photons, as well as their extended families, producing in harmony every ingredient needed to cook up the knowable world. Avoiding the infinitely small meant avoiding a variety of catastrophes. For one, quantum uncertainty couldn’t rip space-time to shreds. At last, it seemed, here was a workable theory of quantum gravity.

Even more beautiful than the story told in words was the elegance of the math behind it, which had the power to make some physicists ecstatic.

To be sure, the theory came with unsettling implications. The strings were too small to be probed by experiment and lived in as many as 11 dimensions of space. These dimensions were folded in on themselves — or “compactified” — into complex origami shapes. No one knew just how the dimensions were compactified — the possibilities for doing so appeared to be endless — but surely some configuration would turn out to be just what was needed to produce familiar forces and particles.

For a time, many physicists believed that string theory would yield a unique way to combine quantum mechanics and gravity. “There was a hope. A moment,” said David Gross, an original player in the so-called Princeton String Quartet, a Nobel Prize winner and permanent member of the Kavli Institute for Theoretical Physics at the University of California, Santa Barbara. “We even thought for a while in the mid-’80s that it was a unique theory.”

And then physicists began to realize that the dream of one singular theory was an illusion. The complexities of string theory, all the possible permutations, refused to reduce to a single one that described our world. “After a certain point in the early ’90s, people gave up on trying to connect to the real world,” Gross said. “The last 20 years have really been a great extension of theoretical tools, but very little progress on understanding what’s actually out there.”

Many, in retrospect, realized they had raised the bar too high. Coming off the momentum of completing the solid and powerful “standard model” of particle physics in the 1970s, they hoped the story would repeat — only this time on a mammoth, all-embracing scale. “We’ve been trying to aim for the successes of the past where we had a very simple equation that captured everything,” said Robbert Dijkgraaf, the director of the Institute for Advanced Study in Princeton, New Jersey. “But now we have this big mess.”

Like many a maturing beauty, string theory has gotten rich in relationships, complicated, hard to handle and widely influential. Its tentacles have reached so deeply into so many areas in theoretical physics, it’s become almost unrecognizable, even to string theorists. “Things have gotten almost postmodern,” said Dijkgraaf, who is a painter as well as mathematical physicist.

The mathematics that have come out of string theory have been put to use in fields such as cosmology and condensed matter physics — the study of materials and their properties. It’s so ubiquitous that “even if you shut down all the string theory groups, people in condensed matter, people in cosmology, people in quantum gravity will do it,” Dijkgraaf said.

“It’s hard to say really where you should draw the boundary around and say: This is string theory; this is not string theory,” said Douglas Stanford, a physicist at the IAS. “Nobody knows whether to say they’re a string theorist anymore,” said Chris Beem, a mathematical physicist at the University of Oxford. “It’s become very confusing.”

String theory today looks almost fractal. The more closely people explore any one corner, the more structure they find. Some dig deep into particular crevices; others zoom out to try to make sense of grander patterns. The upshot is that string theory today includes much that no longer seems stringy. Those tiny loops of string whose harmonics were thought to breathe form into every particle and force known to nature (including elusive gravity) hardly even appear anymore on chalkboards at conferences. At last year’s big annual string theory meeting, the Stanford University string theorist Eva Silverstein was amused to find she was one of the few giving a talk “on string theory proper,” she said. A lot of the time she works on questions related to cosmology.

Even as string theory’s mathematical tools get adopted across the physical sciences, physicists have been struggling with how to deal with the central tension of string theory: Can it ever live up to its initial promise? Could it ever give researchers insight into how gravity and quantum mechanics might be reconciled — not in a toy universe, but in our own?

“The problem is that string theory exists in the landscape of theoretical physics,” said Juan Maldacena, a mathematical physicist at the IAS and perhaps the most prominent figure in the field today. “But we still don’t know yet how it connects to nature as a theory of gravity.” Maldacena now acknowledges the breadth of string theory, and its importance to many fields of physics — even those that don’t require “strings” to be the fundamental stuff of the universe — when he defines string theory as “Solid Theoretical Research in Natural Geometric Structures.”

An Explosion of Quantum Fields

One high point for string theory as a theory of everything came in the late 1990s, when Maldacena revealed that a string theory including gravity in five dimensions was equivalent to a quantum field theory in four dimensions. This “AdS/CFT” duality appeared to provide a map for getting a handle on gravity — the most intransigent piece of the puzzle — by relating it to good old well-understood quantum field theory.

This correspondence was never thought to be a perfect real-world model. The five-dimensional space in which it works has an “anti-de Sitter” geometry, a strange M.C. Escher-ish landscape that is not remotely like our universe.

But researchers were surprised when they dug deep into the other side of the duality. Most people took for granted that quantum field theories — “bread and butter physics,” Dijkgraaf calls them — were well understood and had been for half a century. As it turned out, Dijkgraaf said, “we only understand them in a very limited way.”

These quantum field theories were developed in the 1950s to unify special relativity and quantum mechanics. They worked well enough for long enough that it didn’t much matter that they broke down at very small scales and high energies. But today, when physicists revisit “the part you thought you understood 60 years ago,” said Nima Arkani-Hamed, a physicist at the IAS, you find “stunning structures” that came as a complete surprise. “Every aspect of the idea that we understood quantum field theory turns out to be wrong. It’s a vastly bigger beast.”

Researchers have developed a huge number of quantum field theories in the past decade or so, each used to study different physical systems. Beem suspects there are quantum field theories that can’t be described even in terms of quantum fields. “We have opinions that sound as crazy as that, in large part, because of string theory.”

This virtual explosion of new kinds of quantum field theories is eerily reminiscent of physics in the 1930s, when the unexpected appearance of a new kind of particle — the muon — led a frustrated I.I. Rabi to ask: “Who ordered that?” The flood of new particles was so overwhelming by the 1950s that it led Enrico Fermi to grumble: “If I could remember the names of all these particles, I would have been a botanist.”

Physicists began to see their way through the thicket of new particles only when they found the more fundamental building blocks making them up, like quarks and gluons. Now many physicists are attempting to do the same with quantum field theory. In their attempts to make sense of the zoo, many learn all they can about certain exotic species.

Conformal field theories (the right hand of AdS/CFT) are a starting point. You start with a simplified type of quantum field theory that behaves the same way at small and large distances, said David Simmons-Duffin, a physicist at the IAS. If these specific kinds of field theories could be understood perfectly, answers to deep questions might become clear. “The idea is that if you understand the elephant’s feet really, really well, you can interpolate in between and figure out what the whole thing looks like.”

Like many of his colleagues, Simmons-Duffin says he’s a string theorist mostly in the sense that it’s become an umbrella term for anyone doing fundamental physics in underdeveloped corners. He’s currently focusing on a physical system that’s described by a conformal field theory but has nothing to do with strings. In fact, the system is water at its “critical point,” where the distinction between gas and liquid disappears. It’s interesting because water’s behavior at the critical point is a complicated emergent system that arises from something simpler. As such, it could hint at dynamics behind the emergence of quantum field theories.

Beem focuses on supersymmetric field theories, another toy model, as physicists call these deliberate simplifications. “We’re putting in some unrealistic features to make them easier to handle,” he said. Specifically, they are amenable to tractable mathematics, which “makes it so a lot of things are calculable.”

Toy models are standard tools in most kinds of research. But there’s always the fear that what one learns from a simplified scenario does not apply to the real world. “It’s a bit of a deal with the devil,” Beem said. “String theory is a much less rigorously constructed set of ideas than quantum field theory, so you have to be willing to relax your standards a bit,” he said. “But you’re rewarded for that. It gives you a nice, bigger context in which to work.”

It’s the kind of work that makes people such as Sean Carroll, a theoretical physicist at the California Institute of Technology, wonder if the field has strayed too far from its early ambitions — to find, if not a “theory of everything,” at least a theory of quantum gravity. “Answering deep questions about quantum gravity has not really happened,” he said. “They have all these hammers and they go looking for nails.” That’s fine, he said, even acknowledging that generations might be needed to develop a new theory of quantum gravity. “But it isn’t fine if you forget that, ultimately, your goal is describing the real world.”

It’s a question he has asked his friends. Why are they investigating detailed quantum field theories? “What’s the aspiration?” he asks. Their answers are logical, he says, but steps removed from developing a true description of our universe.

Instead, he’s looking for a way to “find gravity inside quantum mechanics.” A paper he recently wrote with colleagues claims to take steps toward just that. It does not involve string theory.

The Broad Power of Strings

Perhaps the field that has gained the most from the flowering of string theory is mathematics itself. Sitting on a bench beside the IAS pond while watching a blue heron saunter in the reeds, Clay Córdova, a researcher there, explained how what seemed like intractable problems in mathematics were solved by imagining how the question might look to a string. For example, how many spheres could fit inside a Calabi-Yau manifold — the complex folded shape expected to describe how spacetime is compactified? Mathematicians had been stuck. But a two-dimensional string can wiggle around in such a complex space. As it wiggled, it could grasp new insights, like a mathematical multidimensional lasso. This was the kind of physical thinking Einstein was famous for: thought experiments about riding along with a light beam revealed E=mc². Imagining falling off a building led to his biggest eureka moment of all: Gravity is not a force; it’s a property of space-time.

Using the physical intuition offered by strings, physicists produced a powerful formula for getting the answer to the embedded sphere question, and much more. “They got at these formulas using tools that mathematicians don’t allow,” Córdova said. Then, after string theorists found an answer, the mathematicians proved it on their own terms. “This is a kind of experiment,” he explained. “It’s an internal mathematical experiment.” Not only was the stringy solution not wrong, it led to Fields Medal-winning mathematics. “This keeps happening,” he said.

String theory has also made essential contributions to cosmology. The role that string theory has played in thinking about mechanisms behind the inflationary expansion of the universe — the moments immediately after the Big Bang, where quantum effects met gravity head on — is “surprisingly strong,” said Silverstein, even though no strings are attached.

Still, Silverstein and colleagues have used string theory to discover, among other things, ways to see potentially observable signatures of various inflationary ideas. The same insights could have been found using quantum field theory, she said, but they weren’t. “It’s much more natural in string theory, with its extra structure.”

Inflationary models get tangled in string theory in multiple ways, not least of which is the multiverse — the idea that ours is one of a perhaps infinite number of universes, each created by the same mechanism that begat our own. Between string theory and cosmology, the idea of an infinite landscape of possible universes became not just acceptable, but even taken for granted by a large number of physicists. The selection effect, Silverstein said, would be one quite natural explanation for why our world is the way it is: In a very different universe, we wouldn’t be here to tell the story.

This effect could be one answer to a big problem string theory was supposed to solve. As Gross put it: “What picks out this particular theory” — the Standard Model — from the “plethora of infinite possibilities?”

Silverstein thinks the selection effect is actually a good argument for string theory. The infinite landscape of possible universes can be directly linked to “the rich structure that we find in string theory,” she said — the innumerable ways that string theory’s multidimensional space-time can be folded in upon itself.

Building the New Atlas

At the very least, the mature version of string theory — with its mathematical tools that let researchers view problems in new ways — has provided powerful new methods for seeing how seemingly incompatible descriptions of nature can both be true. The discovery of dual descriptions of the same phenomenon pretty much sums up the history of physics. A century and a half ago, James Clerk Maxwell saw that electricity and magnetism were two sides of a coin. Quantum theory revealed the connection between particles and waves. Now physicists have strings.

“Once the elementary things we’re probing spaces with are strings instead of particles,” said Beem, the strings “see things differently.” If it’s too hard to get from A to B using quantum field theory, reimagine the problem in string theory, and “there’s a path,” Beem said.

In cosmology, string theory “packages physical models in a way that’s easier to think about,” Silverstein said. It may take centuries to tie together all these loose strings to weave a coherent picture, but young researchers like Beem aren’t bothered a bit. His generation never thought string theory was going to solve everything. “We’re not stuck,” he said. “It doesn’t feel like we’re on the verge of getting it all sorted, but I know more each day than I did the day before – and so presumably we’re getting somewhere.”

Stanford thinks of it as a big crossword puzzle. “It’s not finished, but as you start solving, you can tell that it’s a valid puzzle,” he said. “It’s passing consistency checks all the time.”

“Maybe it’s not even possible to capture the universe in one easily defined, self-contained form, like a globe,” Dijkgraaf said, sitting in Robert Oppenheimer’s many windowed office from when he was Einstein’s boss, looking over the vast lawn at the IAS, the pond and the woods in the distance. Einstein, too, tried and failed to find a theory of everything, and it takes nothing away from his genius.

“Perhaps the true picture is more like the maps in an atlas, each offering very different kinds of information, each spotty,” Dijkgraaf said. “Using the atlas will require that physics be fluent in many languages, many approaches, all at the same time. Their work will come from many different directions, perhaps far-flung.”

He finds it “totally disorienting” and also “fantastic.”

Arkani-Hamed believes we are in the most exciting epoch of physics since quantum mechanics appeared in the 1920s. But nothing will happen quickly. “If you’re excited about responsibly attacking the very biggest existential physics questions ever, then you should be excited,” he said. “But if you want a ticket to Stockholm for sure in the next 15 years, then probably not.”

This article was reprinted on TheAtlantic.com.