Crucial Computer Program for Particle Physics at Risk of Obsolescence

Recently, I watched a fellow particle physicist talk about a calculation he had pushed to a new height of precision. His tool? A 1980s-era computer program called FORM.

Particle physicists use some of the longest equations in all of science. To look for signs of new elementary particles in collisions at the Large Hadron Collider, for example, they draw thousands of pictures called Feynman diagrams that depict possible collision outcomes, each one encoding a complicated formula that can be millions of terms long. Summing formulas like these with pen and paper is impossible; even adding them with computers is a challenge. The algebra rules we learn in school are fast enough for homework, but for particle physics they are woefully inefficient.

Programs called computer algebra systems strive to handle these tasks. And if you want to solve the biggest equations in the world, for 33 years one program has stood out: FORM.

Developed by the Dutch particle physicist Jos Vermaseren, FORM is a key part of the infrastructure of particle physics, necessary for the hardest calculations. However, as with surprisingly many essential pieces of digital infrastructure, FORM’s maintenance rests largely on one person: Vermaseren himself. And at 73, Vermaseren has begun to step back from FORM development. Due to the incentive structure of academia, which prizes published papers, not software tools, no successor has emerged. If the situation does not change, particle physics may be forced to slow down dramatically.

FORM got its start in the mid-1980s, when the role of computers was changing rapidly. Its predecessor, a program called Schoonschip created by Martinus Veltman, was released as a specialized chip that you plugged into the side of an Atari computer. Vermaseren wanted to make a more accessible program that could be downloaded by universities around the world. He began to program it in the computer language FORTRAN, which stands for Formula Translation. The name FORM was a riff on that. (He later switched to a programming language called C.) Vermaseren released his software in 1989. By the early ’90s, over 200 institutions around the world had downloaded it, and the number kept climbing.

Since 2000, a particle physics paper that cites FORM has been published every few days, on average. “Most of the [high-precision] results that our group obtained in the past 20 years were heavily based on FORM code,” said Thomas Gehrmann, a professor at the University of Zurich.

Some of FORM’s popularity came from specialized algorithms that were built up over the years, such as a trick for quickly multiplying certain pieces of a Feynman diagram, and a procedure for rearranging equations to have as few multiplications and additions as possible. But FORM’s oldest and most powerful advantage is how it handles memory.

Just as humans have two types of memory, short-term and long-term, computers have two types: main and external. Main memory — your computer’s RAM — is easy to access on the fly but limited in size. External memory devices like hard disks and solid-state drives hold much more information but are slower. To solve a long equation, you need to store it in main memory so you can easily work with it.

In the ’80s, both types of memory were limited. “FORM was built in a time when there was almost no memory, and also no disk space — basically there was nothing,” said Ben Ruijl, a former student of Vermaseren’s and FORM developer who is now a postdoctoral researcher at the Swiss Federal Institute of Technology Zurich. This posed a challenge: Equations were too long for main memory to handle. To calculate one, your operating system needed to treat your hard disk as if it was main memory too. The operating system, not knowing how big to expect your equation to be, would store the data in a collection of “pages” on the hard disk, frequently switching between them as different pieces were needed — an inefficient process called swapping.

FORM bypasses swapping and uses its own technique. When you work with an equation in FORM, the program assigns each term a fixed amount of space on the hard disk. This technique lets the software more easily keep track of where the pieces of an equation are. It also makes it easy to bring those pieces back to main memory when they are needed without accessing the rest.

Memory has grown since FORM’s early days, from 128 kilobytes of RAM in the Atari 130XE in 1985 to 128 gigabytes of RAM in my souped-up desktop — a millionfold improvement. But the tricks Vermaseren developed remain crucial. As particle physicists pore through petabytes of data from the Large Hadron Collider to search for evidence of new particles, their need for precision, and thus the length of their equations, grows longer.

“These things will forever stay relevant, however large the memory grows, because there’s always a physics problem that can push it beyond the size of the memory,” said Ruijl.

Computer capabilities have grown roughly exponentially, doubling about every two years. But there are faster forms of growth than exponential growth. Consider the task of writing three letters — a, b and c — in all possible orders. There are three choices for the first letter (a, b or c), two for the second, and one for the third. The problem scales as a factorial, a mathematical relationship that grows even faster than exponential growth. Factorials show up often when you try to count possible combinations of things, such as all the different Feynman diagrams you can draw for a set of colliding particles. The factorial growth of these particle physics calculations outpaces the exponential growth of computing power.

As crucial as software like FORM is for physics, the effort to develop it is often undervalued. Vermaseren was lucky in that he had a permanent position at the National Institute for Subatomic Physics in the Netherlands, and a boss who appreciated the project. But such luck is hard to come by. Stefano Laporta, an Italian physicist who developed a crucial simplification algorithm for the field, has spent most of his career without funding for students or equipment. Universities tend to track scientists’ publication records, which means those who work on critical infrastructure are often passed over for hiring or tenure.

“I have seen over the years, consistently, that people who spend a lot of time on computers don’t get a tenure job in physics,” said Vermaseren.

“It’s more prestigious, maybe, to actually produce physical results than to work on tools,” said Ruijl.

While a few younger physicists like Ruijl work on FORM sporadically, for their careers’ sake they need to spend most of their time on other research. This leaves much of the responsibility for developing FORM in the hands of Vermaseren, who is now mostly retired.

Without ongoing development, FORM will get less and less usable — only able to interact with older computer code, and not aligned with how today’s students learn to program. Experienced users will stick with it, but younger researchers will adopt alternative computer algebra programs like Mathematica that are more user-friendly but orders of magnitude slower. In practice, many of these physicists will decide that certain problems are off-limits — too difficult to handle. So particle physics will stall, with only a few people able to work on the hardest calculations.

In April, Vermaseren is holding a summit of FORM users to plan for the future. They will discuss how to keep FORM alive: how to maintain and extend it, and how to show a new generation of students just how much it can do. With luck, hard work and funding, they may preserve one of the most powerful tools in physics.