A Design Dilemma Solved, Minus Designs

In 1850, the Reverend Thomas Kirkman, rector of the parish of Croft-with-Southworth in Lancashire, England, posed an innocent-looking puzzle in the Lady’s and Gentleman’s Diary, a recreational mathematics journal:

“Fifteen young ladies in a school walk out three abreast for seven days in succession: it is required to arrange them daily, so that no two shall walk twice abreast.” (By “abreast,” Kirkman meant “in a group,” so the girls are walking out in groups of three, and each pair of girls should be in the same group just once.)

The Nine Schoolgirls Challenge:

Solve a variation of Thomas Kirkman’s puzzle by arranging nine girls in walking groups. And think fast — the clock is ticking.

Pull out a pencil and paper, and you’ll quickly find that the problem is harder than it looks: After arranging the schoolgirls for the first two or three days, you’ll almost inevitably have painted yourself into a corner, and have to undo your work.

The puzzle tantalized readers with its simplicity, and in the years following its publication it went viral, in a slow, modestly Victorian sort of way. It generated solutions from amateurs (here’s one of seven solutions) and papers by distinguished mathematicians, and was even turned into a verse by “a lady,” that begins:

A governess of great renown,
Young ladies had fifteen,
Who promenaded near the town,
Along the meadows green.

While Kirkman later bemoaned the fact that his weightier mathematical contributions had been eclipsed by the popularity of this humble brainteaser, he was quick to defend his territory when another prominent mathematician, James Joseph Sylvester, claimed to have created the problem “which has since become so well-known, and fluttered so many a gentle bosom.”

The puzzle may seem like an amusing game (try a simpler version here), but its publication helped launch a field of mathematics called combinatorial design theory that now fills gigantic handbooks. What started as an assortment of conundrums about how to arrange people into groups — or “designs,” as these arrangements came to be called — has since found applications in experiment design, error-correcting codes, cryptography, tournament brackets and even the lottery.

Yet for more than 150 years after Kirkman circulated his schoolgirl problem, the most fundamental question in the field remained unanswered: Do such puzzles usually have solutions? Kirkman’s puzzle is a prototype for a more general problem: If you have n schoolgirls, can you create groups of size k such that each smaller set of size t appears in just one of the larger groups? Such an arrangement is called an (n, k, t) design. (Kirkman’s setup has the additional wrinkle that the groups must be sortable into “days.”)

It’s easy to see that not all choices of n, k and t will work. If you have six schoolgirls, for instance, you can’t make a collection of schoolgirl triples in which every possible pair appears exactly once: Each triple that included “Annabel” would contain two pairs involving her, but Annabel belongs to five pairs, and five is not divisible by two. Many combinations of n, k and t are instantly ruled out by these sorts of divisibility obstacles.

For the parameters that aren’t ruled out, there’s no royal road to finding designs. In many cases, mathematicians have found designs, through a combination of brute force and algebraic methods. But design theorists have also found examples of parameters, such as (43, 7, 2), that have no designs even though all the divisibility requirements check out. Are such cases the exception, mathematicians wondered, or the rule? “It was one of the most famous problems in combinatorics,” said Gil Kalai, a mathematician at the Hebrew University of Jerusalem. He recalls debating the question with a colleague a year and a half ago, and concluding that “we’ll never know the answer, because it’s clearly too hard.”

Just two weeks later, however, a young mathematician named Peter Keevash, of the University of Oxford, proved Kalai wrong. In January 2014, Keevash established that, apart from a few exceptions, designs will always exist if the divisibility requirements are satisfied. In a second paper posted this April on the scientific preprint site arxiv.org, Keevash showed how to count the approximate number of designs for given parameters. This number grows exponentially — for example, there are more than 11 billion ways to arrange 19 schoolgirls into triples so that each pair appears once.

The result is “a bit of an earthquake as far as design theory is concerned,” said Timothy Gowers, a mathematician at the University of Cambridge. The method of the proof, which combines design theory with probability, is something no one expected to work, he said. “It’s a big surprise, what Keevash did.”

Winning Big

Mathematicians realized in the early days of design theory that the field was intimately connected with certain branches of algebra and geometry. For instance, geometric structures called “finite projective planes” — collections of points and lines analogous to those in paintings that use perspective — are really just designs in disguise. The smallest such geometry, a collection of seven points called the Fano plane , gives rise to a (7, 3, 2) design: Each line contains exactly three points, and each pair of points appears in exactly one line. Such connections gave mathematicians a geometric way to generate specific designs.

In the 1920s, the renowned statistician Ronald Fisher showed how to use designs to set up agricultural experiments in which several types of plants had to be compared across different experimental conditions. Today, said Charles Colbourn, a computer scientist at Arizona State University in Tempe, “one of the main things [experiment-planning software] does is construct designs.”

Starting in the 1930s, designs also became widely used to create error-correcting codes, systems that communicate accurately even when information must be sent through noisy channels. Designs translate neatly into error-correcting codes, since they create sets (groups of schoolgirls) that are very different from each other — for instance, in the original schoolgirl problem, no two of the schoolgirl triples contain more than a single girl in common. If you use the schoolgirl groups as your “code words,” then if there’s a transmission error as you are sending one of the code words, you can still figure out which one was sent, since only one code word will be close to the garbled transmission. The Hamming code, one of the most famous early error-correcting codes, is essentially equivalent to the (7, 3, 2) Fano plane design, and another code related to designs was used to encode pictures of Mars that the Mariner 9 probe sent back to Earth in the early 1970s. “Some of the most beautiful codes are ones that are constructed from designs,” Colbourn said.

Design theory may even have been used by betting cartels that made millions of dollars off of Massachusetts’ poorly designed Cash WinFall lottery between 2005 and 2011. That lottery involved choosing six numbers out of 46 choices; tickets won a jackpot if they matched all six numbers, and smaller prizes if they matched five out of six numbers.

There are more than 9 million possible ways to pick six numbers out of 46, so buying tickets with every possible combination would cost far more than the game’s typical jackpot. A number of groups realized, however, that buying hundreds of thousands of tickets would enable them to turn a profit by scooping up many of the smaller prizes. Arguably the best assortment of tickets for such a strategy is a (46, 6, 5) design, which creates tickets of six numbers such that every set of five numbers appears exactly once, guaranteeing either the jackpot or every possible five-number prize.

No one has found a (46, 6, 5) design so far, Colbourn said, but designs exist that are close enough to be useful. Did any of the betting cartels use such a design “to siphon money from the Lottery at no risk to themselves?” wrote Jordan Ellenberg, a mathematician at the University of Wisconsin, Madison, who discussed the Cash WinFall lottery in his book How Not to Be Wrong. If they didn’t, Ellenberg wrote, they probably should have.

It would be hard to make a complete list of the applications of designs, Colbourn said, because new ones are constantly being discovered. “I keep being surprised at how many quite different places designs arise, especially when you least expect them,” he said.

A Perfect Design

As the number of design applications exploded, mathematicians filled reference books with lists of designs that might someday prove useful. “We have tables that say ‘For this set of parameters, 300,000 designs are known,’” said Colbourn, a co-editor of the 1,016-page Handbook of Combinatorial Designs.

Despite the abundance of examples, however, mathematicians struggled to get a handle on just how often designs should exist. The only case they understood thoroughly was the one in which the smallest parameter, t, equals 2: Richard Wilson, of the California Institute of Technology in Pasadena, showed in the mid-1970s that when t = 2, for any k there is at most a finite number of exceptions — values of n that satisfy the divisibility rules but don’t have designs.

But for t greater than 2, no one knew whether designs should usually exist — and for values of t greater than 5, they couldn’t even find a single example of a design. “There were people who felt strongly that [designs] would exist, and others who felt strongly that it’s too much to ask for,” Colbourn said.

In 1985, Vojtěch Rödl of Emory University in Atlanta offered mathematicians a consolation prize: He proved that it’s almost always possible to make a good approximate design — one that perhaps is missing a small fraction of the sets you want, but not many. Rödl’s approach uses a random process to gradually build up the collection of sets — a procedure that came to be known as the Rödl nibble, because, as Keevash put it, “instead of trying to swallow everything at once, you just take a nibble.”

Since then, the Rödl nibble has become a widely used tool in combinatorics, and has even been used in number theory. Last year, for example, mathematicians used it to help establish how far apart prime numbers can be.

But mathematicians agreed that the nibble wouldn’t be useful for attempts to make perfect designs. After all, at the end of Rödl’s procedure, you will typically have missed a small fraction of the smaller sets you need. To make a perfect design, you’d need to add in some additional larger groups that cover the missing sets. But unless you’re very lucky, those new larger groups are going to overlap with some of the groups that are already in your design, sending new errors cascading through your system.

Designs just didn’t seem to have the kind of flexibility that would allow a random approach to work. It seemed “obviously impossible,” Gowers said, that an approach like Rödl’s could be used to make perfect designs.

Last year, however — nearly three decades after Rödl’s work — Keevash showed that it is possible to control the cascade of errors by using an approach that marries flexibility and rigidity. Keevash modified Rödl’s construction by starting off the nibble with a specific collection of schoolgirl groups, called a “template,” that has particularly nice algebraic properties. At the end of the nibble, there will be errors to correct, but once the errors propagate into the template, Keevash showed, they can almost always be fixed there in a finite number of steps, producing a perfect design. “The full proof is extremely delicate and it is a phenomenal achievement,” wrote Ross Kang, of Radboud University in the Netherlands.

“I think a few years ago, nobody thought that a proof was on the horizon,” Colbourn said. “It’s an extraordinary breakthrough.”

For pure mathematicians, Keevash’s result is in a sense the end of the story: It establishes that for any parameters t and k, all values of n that fit the divisibility conditions will have a design, apart from at most a finite number of exceptions. “It sort of kills off a whole class of problems,” Gowers said.

But Keevash’s result leaves many mysteries unsolved for people who care about actual designs. In theory, his template-nibble approach could be used to create designs, but for now it’s unclear how large n has to be for his method to work, or how long an algorithm based on his method would take to run. And while Keevash has proved that designs almost always exist, his result doesn’t say whether a design will exist for any particular set of parameters you might care about. “People will presumably still work on this for generations,” Wilson said.

Still, Keevash’s result will shift the mindset of mathematicians who are trying to find designs, Colbourn said. “Before, it wasn’t clear whether the focus should be on constructing designs or proving they don’t exist,” he said. “Now at least we know the effort should focus on constructing them.”

And the shortage of information about specific designs leaves plenty of fun puzzles for recreational mathematicians to solve. So in the spirit of Kirkman, we will leave the gentle reader with another brainteaser, a slight variation on the schoolgirl puzzle devised in 1917 by the British puzzle aficionado Henry Ernest Dudeney and later popularized by Martin Gardner: Nine prisoners are taken outdoors for exercise in rows of three, with each adjacent pair of prisoners linked by handcuffs, on each of the six weekdays (back in Dudeney’s less enlightened times, Saturday was still a weekday). Can the prisoners be arranged over the course of the six days so that each pair of prisoners shares handcuffs exactly once?

Dudeney wrote that this puzzle is “quite a different problem from the old one of the Fifteen Schoolgirls, and it will be found to be a fascinating teaser and amply repay for the leisure time spent on its solution.” Happy solving!

Editor’s note: The first reader to submit a correct solution to the nine prisoners problem in the comments section will receive a Quanta Magazine T-shirt.

 This article was reprinted on Wired.com.