Why Claude Shannon Would Have Been Great at Wordle

If you’ve interacted with another human being this year, you’ve probably heard of Wordle, the addictive word-guessing game the coder Josh Wardle created for his partner and then sold to The New York Times for over $1 million. You might even be one of the millions who enjoy guessing those five-letter words seemingly selected with just the right balance of difficulty and solvability.

Perhaps you have a favorite first word that helps you solve the puzzle in fewer than the six allowed guesses. Or maybe you like to mix it up at the start and play your hunches. However you approach this word game, knowing a bit about a mathematical field called information theory can help you achieve your best scores.

To see how, let’s say you open up Wordle and your first guess at the secret word, after a big breakfast, is BLOAT. Wordle shows you this:

Yellow tells you that A and T are in the secret word but in the wrong positions. Knowing the word contains an A and a T, and running late for school or work, you guess WATCH and get lucky.

The green letters are in the secret word and in those exact positions. You’ve almost got it! So, what’s your next guess? How you continue says something about you both as a Wordle player and as an information theorist.

One approach is to guess a word like MATCH, which could be the answer. But, strange as it may seem, a better strategy is to guess CHIMP. Even though CHIMP can’t be the secret word, it’s the perfect move according to information theory, the field pioneered by Claude Shannon in the 1940s that laid the foundation for the digital revolution. Let’s see why.

Shannon was interested in quantifying the amount of information contained in messages across different contexts, from communication over telephone lines (he worked at Bell Labs, the famous research branch of the American phone company) to the knowledge stored in DNA (he wrote his doctoral thesis on genetics). In defining the concept of “information,” Shannon started from a few basic mathematical principles. First, the amount of information should be inversely related to predictability: A rare event should be more informative than an expected one. And second, information should add up: The amount of information in two messages should be related to the sum of the information from each individual message.

We’ll get to the details of Shannon’s definition of information in a moment, but first let’s finish up our Wordle game. Here’s everything from Wordle’s word list that ends in ATCH: BATCH, CATCH, HATCH, LATCH, MATCH, PATCH and WATCH.

From our first guess, BLOAT, we know the secret word doesn’t contain a B or an L, so that eliminates BATCH and LATCH. We also know WATCH isn’t the word, so we’ve narrowed the list of possible secret words down to: CATCH, HATCH, MATCH or PATCH.

If we try MATCH we might get lucky and win the game. But what if MATCH isn’t the word? We can try CATCH, HATCH and PATCH and eventually win the game by process of elimination, but it could take as many as four guesses to get there.

Now consider what happens when we guess CHIMP. If M or P in CHIMP comes back yellow, we know the word is MATCH or PATCH, respectively. If the C in CHIMP comes back green, we know the word is CATCH. And if none of those things happen, the only other possibility is HATCH. By guessing CHIMP, we’re guaranteed to get the word on our next guess.

Shannon’s second principle helps us understand why CHIMP is the right move. It’s not that MATCH is a bad guess. It’s more that if it isn’t the answer, that’s the only information we gain. Guessing CHIMP leverages the additivity of information from the letters C, H, M and P to give us everything we need to know to solve the puzzle.

Now that our Wordle streak is safe, let’s dig deeper into Shannon’s definition of information. The mathematical principles he started with led him to the following mathematical formula:

$latex I = \log_{2}\frac{1}{p}$.

Here I stands for information, measured in what Shannon deemed “bits”; p is the probability of the event whose information content you are quantifying; and $latex\log_{2}$ represents the base-2 logarithm function. Logarithms tell you the exponent of a number relative to a certain base: For example, $latex\log_{2}16 = 4$ because $latex 2^4 = 16$, and  $latex\log_{2}32 = 5$ because $latex2^5 = 32$. You’d need a calculator (or a logarithm table) to figure out that $latex\log_{2}25 \approx 4.643856…$, but you know it’s between 4 and 5 because $latex16<25<32$. (You can choose any base of the logarithm function to quantify information, but base 2 is standard in a world of digital information stored in binary, or base-2, numbers.)

Let’s see how this definition of information follows from the two guiding principles mentioned above. First, because of the way logarithms work, the quantity

$latex I = \log_{2}\frac{1}{p}$

gets bigger as the probability of an event gets smaller. For example, an event that happens one out of every two times contains one bit of information, because $latexp=\frac{1}{2}$ and so

$latex\log_{2}\frac{1}{\frac{1}{2}}=\log_{2}2=1$.

But an event that happens once every 32 times contains five bits of information, since $latexp=\frac{1}{32}$, and so

$latex\log_{2}\frac{1}{\frac{1}{32}}=\log_{2}32=5$.

The less predictable something is, the more information it contains. A digital photo of a polar bear in a snowstorm wouldn’t contain much information because all the pixels would be predictably white. On the other hand, a lot of information would be contained in a picture of two multicolored birds of paradise dancing on the forest floor.

The second principle, the additivity of information, follows from a law of logarithms you might remember learning in algebra class:

$latex\log_{2}AB=\log_{2}A +\log_{2}B$.

In other words, the log of a product is the sum of the numbers’ individual logs. To see the connection to information, imagine that two events occur with individual probabilities $latexp_{1}$ and $latexp_{2}$. If these events are independent — that is, neither event influences the outcome of the other — then the probability of both events happening is the product of those probabilities, $latexp_{1}p_{2}$. For example, each flip of a fair coin is independent of every other flip, so the probability of flipping tails twice in a row on a fair coin is just $latex\frac{1}{2}\times\frac{1}{2}=\frac{1}{4}$, which is the probability of flipping tails on the first toss times the probability of flipping tails on the second.

When two events are independent, and thus don’t influence each other, knowing about one doesn’t give you any information about the other, so it makes sense to add up information in this situation. The total information associated with both events occurring is equal to the sum of the information associated with each event occurring individually. And it’s the clever choice of logarithms in the definition that makes information add up. The probability of the two independent events occurring together is $latexp_{1}p_{2}$, so the associated measure of information is

$latex\log_{2}\frac{1}{p_{1}p_{2}} = \log_{2}\frac{1}{p_{1} }\times \frac{1}{p_{2}}=\log_{2}\frac{1}{p_{1}} + \log_{2}\frac{1}{p_{2}}$,

which is the sum of the information from the individual events.

To see some information theory in action, let’s play a simplified version of Wordle that uses only two-letter words. In this game of “2-Wordle” there are only 16 possible answers: