Where Quantum Probability Comes From

In A Philosophical Essay on Probabilities, published in 1814, Pierre-Simon Laplace introduced a notorious hypothetical creature: a “vast intelligence” that knew the complete physical state of the present universe. For such an entity, dubbed “Laplace’s demon” by subsequent commentators, there would be no mystery about what had happened in the past or what would happen at any time in the future. According to the clockwork universe described by Isaac Newton, the past and future are exactly determined by the present.

Laplace’s demon was never supposed to be a practical thought experiment; the imagined intelligence would have to be essentially as vast as the universe itself. And in practice, chaotic dynamics can amplify tiny imperfections in the initial knowledge of the system into complete uncertainty later on. But in principle, Newtonian mechanics is deterministic.

A century later, quantum mechanics changed everything. Ordinary physical theories tell you what a system is and how it evolves over time. Quantum mechanics does this as well, but it also comes with an entirely new set of rules, governing what happens when systems are observed or measured. Most notably, measurement outcomes cannot be predicted with perfect confidence, even in principle. The best we can do is to calculate the probability of obtaining each possible outcome, according to what’s called the Born rule: The wave function assigns an “amplitude” to each measurement outcome, and the probability of getting that result is equal to the amplitude squared. This feature is what led Albert Einstein to complain about God playing dice with the universe.

Researchers continue to argue over the best way to think about quantum mechanics. There are competing schools of thought, which are sometimes referred to as “interpretations” of quantum theory but are better thought of as distinct physical theories that give the same predictions in the regimes we have tested so far. All of them share the feature that they lean on the idea of probability in a fundamental way. Which raises the question: What is “probability,” really?

Like many subtle concepts, probability starts out with a seemingly straightforward, commonsensical meaning, which becomes trickier the closer we look at it. You flip a fair coin many times; whether it comes up heads or tails on any particular trial is completely unknown, but if we perform many trials we expect to get heads 50% of the time and tails 50% of the time. We therefore say that the probability of obtaining heads is 50%, and likewise for tails.

We know how to handle the mathematics of probability, thanks to the work of the Russian mathematician Andrey Kolmogorov and others. Probabilities are real numbers between zero and one, inclusive; the probabilities of all independent events add up to one; and so on. But that’s not the same as deciding what probability actually is.

There are numerous approaches to defining probability, but we can distinguish between two broad classes. The “objective” or “physical” view treats probability as a fundamental feature of a system, the best way we have to characterize physical behavior. An example of an objective approach to probability is frequentism, which defines probability as the frequency with which things happen over many trials, as in our coin-tossing example.

Alternatively, there are “subjective” or “evidential” views, which treat probability as personal, a reflection of an individual’s credence, or degree of belief, about what is true or what will happen. An example is Bayesian probability, which emphasizes Bayes’ law, a mathematical theorem that tells us how to update our credences as we obtain new information. Bayesians imagine that rational creatures in states of incomplete information walk around with credences for every proposition you can imagine, updating them continually as new data comes in. In contrast with frequentism, in Bayesianism it makes perfect sense to attach probabilities to one-shot events, such as who will win the next election, or even past events that we’re unsure about.

Interestingly, different approaches to quantum mechanics invoke different meanings of probability in central ways. Thinking about quantum mechanics helps illuminate probability, and vice versa. Or, to put it more pessimistically: Quantum mechanics as it is currently understood doesn’t really help us choose between competing conceptions of probability, as every conception has a home in some quantum formulation or other.

Let’s consider three of the leading approaches to quantum theory. There are “dynamical collapse” theories, such as the GRW model proposed in 1985 by Giancarlo Ghirardi, Alberto Rimini and Tullio Weber. There are “pilot wave” or “hidden variable” approaches, most notably the de Broglie-Bohm theory, invented by David Bohm in 1952 based on earlier ideas from Louis de Broglie. And there is the “many worlds” formulation suggested by Hugh Everett in 1957.

Each of these represents a way of solving the measurement problem of quantum mechanics. The problem is that conventional quantum theory describes the state of a system in terms of a wave function, which evolves smoothly and deterministically according to the Schrödinger equation. At least, it does unless the system is being observed; in that case, according to the textbook presentation, the wave function suddenly “collapses” into some particular observational outcome. The collapse itself is unpredictable; the wave function assigns a number to each possible outcome, and the probability of observing that outcome is equal to the value of the wave function squared. The measurement problem is simply: What constitutes a “measurement”? When exactly does it occur? Why are measurements seemingly different from ordinary evolution?

Dynamical-collapse theories offer perhaps the most straightforward resolution to the measurement problem. They posit that there is a truly random component to quantum evolution, according to which every particle usually obeys the Schrödinger equation, but occasionally its wave function will spontaneously localize at some position in space. Such collapses are so rare that we would never observe one for a single particle, but in a macroscopic object made of many particles, collapses happen all the time. This prevents macroscopic objects — like the cat in Schrödinger’s infamous thought experiment — from evolving into an observable superposition. All the particles in a large system will be entangled with each other, so that when just one of them localizes in space, the rest are brought along for the ride.

Probability in such models is fundamental and objective. There is absolutely nothing about the present that precisely determines the future. Dynamical-collapse theories fit perfectly into an old-fashioned frequentist view of probability. What happens next is unknowable, and all we can say is what the long-term frequency of different outcomes will be. Laplace’s demon wouldn’t be able to exactly predict the future, even if it knew the present state of the universe exactly.

Pilot-wave theories tell a very different story. Here, nothing is truly random; the quantum state evolves deterministically, just as the classical state did for Newton. The new element is the concept of hidden variables, such as the actual positions of particles, in addition to the traditional wave function. The particles are what we actually observe, while the wave function serves merely to guide them.

In a sense, pilot-wave theories bring us back to the clockwork universe of classical mechanics, but with an important twist: When we’re not making an observation, we don’t, and can’t, know the actual values of the hidden variables. We can prepare a wave function so that we know it exactly, but we only learn about the hidden variables by observing them. The best we can do is to admit our ignorance and introduce a probability distribution over their possible values.

Probability in pilot-wave theories, in other words, is entirely subjective. It characterizes our knowledge, not an objective frequency of occurrences over time. A full-powered Laplace demon that knew both the wave function and all the hidden variables could predict the future exactly, but a hobbled version that only knew the wave function would still have to make probabilistic predictions.

Then we have many-worlds. This is my personal favorite approach to quantum mechanics, but it’s also the one for which it is most challenging to pinpoint how and why probability enters the game.

Many-worlds quantum mechanics has the simplest formulation of all the alternatives. There is a wave function, and it obeys Schrödinger’s equation, and that’s all. There are no collapses and no additional variables. Instead, we use Schrödinger’s equation to predict what will happen when an observer measures a quantum object in a superposition of multiple possible states. The answer is that the combined system of observer and object evolves into an entangled superposition. In each part of the superposition, the object has a definite measurement outcome and the observer has measured that outcome.

Everett’s brilliant move was simply to say, “And that’s okay” — all we need to do is recognize that each part of the system subsequently evolves separately from all of the others, and therefore qualifies as a separate branch of the wave function, or “world.” The worlds aren’t put in by hand; they were lurking in the quantum formalism all along.

The idea of all those worlds might seem extravagant or distasteful, but those aren’t respectable scientific objections. A more legitimate question is the nature of probability within this approach. In many-worlds, we can know the wave function exactly, and it evolves deterministically. There is nothing unknown or unpredictable. Laplace’s demon could predict the entire future of the universe with perfect confidence. How is probability involved at all?

An answer is provided by the idea of “self-locating,” or “indexical,” uncertainty. Imagine that you are about to measure a quantum system, thus branching the wave function into different worlds (for simplicity, let’s just say there will be two worlds). It doesn’t make sense to ask, “After the measurement, which world will I be on?” There will be two people, one on each branch, both descended from you; neither has a better claim to being “really you” than the other.

But even if both people know the wave function of the universe, there is now something they don’t know: which branch of the wave function they are on. There will inevitably be a period of time after branching occurs but before the observers find out what outcome was obtained on their branch. They don’t know where they are in the wave function. That’s self-locating uncertainty, as first emphasized in the quantum context by the physicist Lev Vaidman.

You might think you could just look at the experimental outcome really quickly, so that there was no noticeable period of uncertainty. But in the real world, the wave function branches incredibly fast, on timescales of 10⁻²¹ seconds or less. That’s far quicker than a signal can even reach your brain. There will always be some period of time when you’re on a certain branch of the wave function, but you don’t know which one.

Can we resolve this uncertainty in a sensible way? Yes, we can, as Charles Sebens and I have argued, and doing so leads precisely to the Born rule: The credence you should attach to being on any particular branch of the wave function is just the amplitude squared for that branch, just as in ordinary quantum mechanics. Sebens and I needed to make a new assumption, which we called the “epistemic separability principle”: Whatever predictions you make for experimental outcomes, they should be unaltered if we only change the wave function for completely separate parts of the system.

Self-locating uncertainty is a different kind of epistemic uncertainty from that featured in pilot-wave models. You can know everything there is to know about the universe, and there’s still something you’re uncertain about, namely where you personally are within it. Your uncertainty obeys the rules of ordinary probability, but it requires a bit of work to convince yourself that there’s a reasonable way to assign numbers to your belief.

You might object that you want to make predictions now, even before branching happens. Then there’s nothing uncertain; you know exactly how the universe will evolve. But included in that knowledge is the conviction that all the future versions of yourself will be uncertain, and they should use the Born rule to assign credences to the various branches they could be on. In that case, it makes sense to act precisely as if you live in a truly stochastic universe, with the frequency of various outcomes given by the Born rule. (David Deutsch and David Wallace have made this argument rigorous using decision theory.)

In one sense, all of these notions of probability can be thought of as versions of self-locating uncertainty. All we have to do is consider the set of all possible worlds — all the different versions of reality one could possibly conceive. Some such worlds obey the rules of dynamical-collapse theories, and each of these is distinguished by the actual sequence of outcomes for all the quantum measurements ever performed. Other worlds are described by pilot-wave theories, and in each one the hidden variables have different values. Still others are many-worlds realities, where agents are uncertain about which branch of the wave function they are on. We might think of the role of probability as expressing our personal credences about which of these possible worlds is the actual one.

The study of probability takes us from coin flipping to branching universes. Hopefully our understanding of this tricky concept will progress hand in hand with our understanding of quantum mechanics itself.