There’s No Need to Choose Between Taking Action and Building Evidence

A recent article by Susan Dominus in the New York Times Magazine described a painful “civil war” that raged in some New York City hospitals, in which two groups of physicians — researchers and critical care doctors — battled over treatments for COVID-19 in the early months of the pandemic. The researchers were pushing for randomized clinical trials to assess the efficacy of various drugs (often, approved drugs that were being prescribed “off label”) since there were no proven treatments for COVID-19. The clinicians, who were desperately trying to save lives during a deluge of critically ill patients, insisted that they had to use their best judgment, that they had no time for studies, and that it would be unethical to withhold a potentially effective drug and allow a patient to receive a placebo, even if there was no evidence about the effectiveness of some of the drugs they were prescribing for the virus. 

In reading the article, it was difficult not to sympathize with the physicians who were treating patients on the frontlines, often placing their own lives at risk. Those of us who live in New York City will never forget the terror of those weeks in March and April, when morgues and funeral homes were overwhelmed, and ambulances seemed to be streaking by our homes every few minutes. The doctors were doing their best in impossible circumstances.  

But, in the end, I believe the researchers were right. Their colleagues were insisting that they should be free to prescribe drugs like hydroxychloroquine, which was later found to be ineffective or even harmful for some patients. Without the ability to run rigorous trials, the world lost a priceless opportunity to build evidence that could have helped thousands of patients in other places where the virus surged later. According to the Times article, the researchers were not necessarily asking the clinicians not to prescribe drugs off-label, but they asked that they do so in the context of clinical trials. 

We face similar challenges in the social policy field. Like past crises, the pandemic has laid bare the weakness of many of our public systems and the brutal inequities that riddle our society. COVID’s public health and economic toll has been borne disproportionately by people of color and those with lower incomes. At the same time, amid the pandemic, nationwide protests have again highlighted stark disparities in the treatment of Black Americans by the criminal justice system, as well as a larger pattern of systemic racial bias.  

Activists and reformers argue that large-scale action is needed now. No more blue-ribbon panels of esteemed academics or small pilot programs. And they are right. 

But, like the critical care physicians, the advocates must be humble enough to admit that we don’t yet know the most effective cures for the illnesses from which our society suffers. Often, policies that seem like obvious solutions produce unintended consequences and don’t end up helping the people they are designed to help.

The answer is not to wait. Yes, let’s move forward with bold reforms. But let’s study them carefully at the same time, even if we can’t always use the most rigorous designs. It can be done. In January 2017, the State of New Jersey implemented a bold, statewide criminal justice reform that virtually eliminated the use of cash bail. MDRC, with funding from Arnold Ventures, was able to use a rigorous quasi-experimental design to evaluate the reform and offer valuable information to inform policymakers in New Jersey and elsewhere.

The cost of research is modest compared to the scale of the problems we face. Let’s ask those who are most directly affected by the changes what questions our studies should be exploring. And let’s be honest about what we learn. If mid-course corrections are needed, let’s make them promptly. And let’s make sure that what we learn quickly gets into the hands of those who need to know.

In short, we don’t need to choose between evidence and action. We need both.

Dan Bloom is Senior Vice President for MDRC, a nonprofit, nonpartisan organization headquartered in New York City that develops and evaluates policies and programs that serve low-income people.