American Scientist

Almost a hundred years ago, Scottish epidemiologists Anderson McKendrick and William Kermack proposed an intricate system of three differential equations—mathematical formulas for changes in variables over time—describing how an infection would spread when individuals from different age groups had different responses to it. This paper laid the groundwork for tens of thousands of articles on the mathematical modeling of infectious diseases, their evolution, their control, and their severity. These models simplify a human population into a series of groups, or compartments, across which individuals move as a function of their epidemic status, age, or position in the social network: susceptible, infectious, exposed, asymptomatic, those who recover, those who die, those at high risk of exposure, and those most likely to infect others. The list of these compartments goes on and on, as the complexity of the transmission increases.

Epidemiologists and mathematical biologists have invested so much time and effort in refining and understanding these models because they have delivered fantastic insights. Will an epidemic spread? What fraction of the population should we vaccinate to stop it? Which age class will be most infected? Can we develop early-warning systems for outbreaks? What proportion of the population will have been infected by the end of the outbreak? These are all questions we can ask of mathematical models, and to which mathematical models answer through ever more equations. This math often comes as a bit of a shock to graduate students taking a class in epidemiology: There is a lot more calculus than they expect.

For all the answers that theoretical models provide, they are a little bit like fables. We do not listen to tales of tortoises and hares hoping to learn something about tortoises and hares; we listen because they make us ponder some more universal questions. As Margaret Atwood wrote in her book The Tent, “No fables are really about animals,” and epidemiologists know that no general mathematical model of an epidemic is really about an epidemic. These models are tools to help us think generally about how infections spread, not tools to help us predict. This point is brought up in the introduction of nearly all disease modeling textbooks, with the understated warning that applying the models as they are introduced will lead to irrelevant conclusions. As discussed at length during the previous SARS epidemics, models can easily ignore public health interventions, and end up futile or misleading.

The COVID-19 pandemic has resulted, predictably, in an explosion of interest in epidemic models. Epidemiologists are trained to make models with cautionary notes about the effects of public health interventions in mind, and their models of the spread of the coronavirus have been essential to saving lives and informing many stakeholders' responses. Yet people with little to no training in epidemiology have also started discussing and even implementing these models, without awareness of the harm they can do. This participation is more tempting than ever before because it has never been so easy to create and share interactive models and data visualizations. By and large, these models are not “wrong.” They are that special brand of “correct” that simply does not translate into “useful,” which is arguably the point of most mathematical models.

“Models that offer a different perspective on expert predictions are only contributing to eroding the public trust in all models.”

• Facebook
• Twitter

The output of epidemiological models can give us better intuitions as to how epidemics work. If we assume that social distancing will decrease the rate of transmission—a reasonable assumption that there is now data to support—we can plug this into a model and see the peak number of infected individuals decrease, the timeline of the infection change, and we can visualize the lowered impact on the healthcare system. This kind of modeling result can help illuminate why these policies are being put into place. But such a result does not translate into a prediction of how many cases of COVID-19 our neighborhood, state, or country will see.

Writing a model to make actual predictions is a far more demanding task. Early models of the 2003 SARS outbreak required an important amount of data, and still resulted in both a wide range of simulated outcomes and vivid discussions about the correct way to formulate them. Predictive models, ones that can be applied to real-world scenarios, comprise the nonfiction genre of mathematical modeling; paradoxically, this task relies less on mathematical prowess and more on real-world understanding. Models tasked with making predictions do not exist in the abstract, ideal universe of differential equations; they exist in the messy world of incomplete data, evolving norms of reporting, confidentiality issues, and the immense uncertainty that comes from addressing a crisis. The ability to navigate equations is required for creating these models, but alone is not sufficient for creating useful models.

This is not to say that models should all be disregarded, because not all models are built in the same way or for the same reasons. Writing a general model always requires assumptions, abstractions, and simplifications that are considered reasonable for the sake of reaching general conclusions. But models that deal with the real world do not have this luxury, and ambiguities must be resolved, parameters must be ground-truthed, and mechanisms need to be explicit. Through such efforts, public health agencies release to the public the range of scenarios they deem possible, ones based on verified data, projections of different contingencies, and intimate knowledge of the health system and its capacity. This work demands that practitioners build a consensus among different stakeholders, express differences of opinions as uncertainty, and weigh different scenarios. Such efforts require communication, without which there can be no models. In a sense, understanding the complexity of building a predictive model offers a powerful heuristic for determining which models can be trusted: It is very unlikely that a lone modeler, no matter their previous successes at predicting sports results, consumer behavior on websites, economic trends, or elections, will be able to understand the entire ecosystem of stakeholders, constraints, and data that are required to make a valuable model, much less access all these data and implement them appropriately in the correct equations.

As a modeler, I understand the urge to turn toward mathematics; models are some of science’s most powerful tools precisely because, through simplification, they bring some semblance of order, predictability, and control to a chaotic world. I understand modeling as therapy. But I also would like to urge greater caution in how we communicate. It is important to be transparent about the limits of models; it is important to be honest about the value of a model as a visual aid or as a policy tool. It is especially important because everyone is looking for answers, and we do not want to lead people to look for them in the wrong place.

“Models that are used by public health agencies to make predictions and recommend policy and interventions are created with a high standard of rigor.”

• Facebook
• Twitter

Models that misrepresent the severity of the COVID-19 pandemic, either by understating its severity or by giving way to complete fatalism, are counterproductive and even harmful. Models that offer a different perspective on expert predictions are only contributing to eroding the public trust in all models, especially as the majority of experts' scenarios already capture a diversity of viewpoints. Faced with a situation as complex as the COVID-19 pandemic, skepticism toward models is wise. We should question the simple answers and resist the temptation to outsmart or outguess the teams who are tasked with producing realistic scenarios. If we have to place our trust somewhere, it should not be in any one model, but in the collective expertise of the team producing it.

Models that are used by public health agencies to make predictions and recommend policy and interventions are created with a high standard of rigor, despite the extreme urgency of the situation. It can be frustrating to not have full access to them, as it can be frustrating to not have a single authoritative repository of models. But this limit comes about because predictive models are finely tailored to local or regional situations: They account for the healthcare system capacity, for the interventions that can be deployed, and for the data that can be collected. The fact that scenarios presented by a single institution can vary by an order of magnitude only reflects an honest communication of the uncertainty, and the fact that they are planning for the worst-case scenario, while working to achieve the best-case one.