When Solid Data Isn’t Enough

I’ll never forget an incident early in my career, working in the Texas legislature, when a pair of academics testified about their statistical analysis for improving the funding distribution for school bus grants to school districts across the state. They used regression analyses to demonstrate a more efficient and fairer way to allocate the aid. But a state legislator was enraged. He asked how he was supposed to explain “this statistical mumbo-jumbo” (actually, his language was more colorful) to parents in his school district when they complained about why their kids couldn’t ride a bus to school.

It’s not just an academic vs. policymaker divide. In looking back, the financial information generated by the 1990 Chief Financial Officers Act and the performance information generated by the 1993 Government Performance and Results Act have had little impact on policy decisions, especially in Congress. The lesson: having technical experts in agencies creating a supply of information is not enough; there needs to be a demand for it, as well.

Up until recently, academics and scientists also felt that creating a supply and improving the quality of studies and research would be enough for policymakers to pay more attention to their work. Their goal was to reduce uncertainty, so legislators and others could make more informed decisions.

Bridging a Culture Gap 

A new pair of articles in a recent issue of Public Administration Review shows that reducing uncertainty isn’t enough, any more than it has been for financial or performance information.

In the first article, researchers Paul Cairney, Kathryn Oliver and Adam Wellstead say:

“Improving the supply of evidence helps reduce scientific and policymaker uncertainty.  However, policymakers also combine their beliefs with limited evidence to reduce ambiguity” when choosing a course of action to solve a problem. The researchers, therefore, explored whether there are solutions to closing the “evidence-policy” gap and “maximize the uptake of scientific evidence within government.”

They argue that academics and scientists need to also factor in coalition building and persuasion. After all, lawmakers want “to reduce ambiguity by establishing a dominant way to ‘frame’ policy problems.” That is, having a point of view, or an ideological solution, simplifies decision making for them. As a result, the researchers say it is important for advocates of scientific evidence to understand this and engage in the process of persuasion, and “combine evidence with simple stories to exploit the emotional and ideological biases of policy makers.”

There are differences between academic and political cultures: language and jargon, longer scientific timescales, low incentives to engage, differing perceptions of scientific knowledge.  Scientists like to constantly challenge evidence to ensure it is robust; policymakers feel a need to create policy certainty and “reconcile evidence with well-established beliefs,” note the researchers.

The researchers think that a potential solution to this culture gap is to use the successful experiences of health and environmental scientists, who have learned to package their evidence in ways that are easy to understand. Oftentimes, they “foster networks with policy makers and encourage academic-practitioner workshops; use intermediaries such as ‘knowledge brokers;’ recognize the importance of timing; and encourage policy maker skills or better government understanding of problems.” The article’s authors found that scientific knowledge only helps to reduce uncertainty. It does not address other dimensions that policy makers consider, “such as the inability to completely separate empirical facts from human values and ways of thinking.”

Using the Tools of Political Policy Experts

The researchers suggest that academics and scientists adopt the tools of political theorists, where “The focus of policy theory is on the links between evidence, persuasion, and framing (in the wider context of a tendency for certain beliefs to dominate discussion).”

They offer the following four strategies to increase demand for scientific evidence:

• Combine facts with emotional appeals.
• Tell simple stories that are easy to understand.
• Interpret new evidence through the lens of the preexisting beliefs.
• Offer policy solutions that are feasible.

They say that scientists have a difficult choice to make: maintain an image of academic objectivity, or engage in advocacy to increase their impact. “Without such advocacy, policymakers will rely on highly biased and persuasive stories from vested interests rather than struggle through inaccessible and equivocal scientific papers,” they wrote.

In conclusion, the article’s authors recommend academics and scientists make efforts to:

• Promote networks between communities of scientists and policymakers to allow both sides to understand how the other camp operates and thinks.
• Reward academics for doing relevant research and engage them in agenda setting.
• Use generalist knowledge brokers who direct questions appropriately and have a good relationship with both policymakers and academics.
• Measure the practical value of academic research on policy.

A Shared Responsibility 

In the second article, Max Arinder, an experienced government executive formerly with the Mississippi Joint Legislative Committee on Performance Evaluation and Expenditure Review, says that getting policy makers to use scientific evidence when making decisions isn’t a one-sided responsibility. Both sides need to be open. He suggests there also needs to be a culture shift in policymaking institutions themselves. “Policymakers often balance evidence, emotions, and beliefs in making a decision,” he said. The question is how to create an optimum balance. “Constituent opinion is a strong voice and will certainly be a part of any legislative arena. But how well informed is that voice?”

Arinder noted that “Just as constituent opinion is important in policymaking, [a sound technical staff] is also an important element in determining the policy culture of a legislature.”

If would help if policymakers developed “a transparent, evidence-informed process that increases public knowledge,” he said. Arinder offers an example: In 2015, Mississippi legislative leaders created a system to review requests for new programs and funding using an “evidence screen.” He said this shows that it’s possible to “make research-sensitive strategies a more important part of our core value system in the selection and funding of policy initiatives and programs.” The Pew Charitable Trusts is sponsoring similar efforts to expand the use of empirical evidence in other state legislatures.

Both articles show how all stakeholders need to engage to shape an environment where the demand for evidence becomes part of the policymaking culture. While decisions may still be based on other considerations, the supporting data will be available and transparent, and hopefully create a better-informed electorate.