Evidence-Based Policy Meets Reality In Federal Agencies

There’s evidence, and then there’s evidence.

As soon as Congress began finalizing last year’s legislation to have agencies embrace evidence in evaluating program success, lawmakers of differing ideologies were already disagreeing over its meaning for real-world decisions involving, for example, efforts to alleviate poverty.

On Wednesday, the Bipartisan Policy Center attempted to describe the shoals on which the movement to incorporate more evidence in funding decisions could founder. Part I of a two-part “Evidence Use in Congress” report identifies 16 barriers in the categories of perception, institutional and systemic.

“Perception barriers relate to how consumers of information view evidence’s utility, including the credibility of the source of the evidence, and the practice of its generation and use,” said the report by center specialists Nick Hart, Edward “Sandy” Davis and Tim Shaw. “Institutional barriers relate to existing organizational structures and whether they are well-designed to incorporate evidence use. Third, systemic barriers are those that arise due to processes and organizational or cultural norms.”

Specific barriers to a culture that welcomes performance data are examined under the headings of perceived usefulness, inconsistent goals, limited credibility, unclear relevancy, collaborative decision-making, insufficient expertise, evolving congressional oversight functions, political design, under-coordination with Executive Branch, timing, undersupply, disincentives, champions, alternative sources, cognitive limits and opacity.

Champions, for example, refers to an absence of legislative leadership support for program evaluation, and notes that a Government Accountability Office report found that nearly 20 percent of federal program managers perceive a lack of congressional support for evaluation as a barrier to their efforts to engage in evidence-building activities.

Politics and budget procedures also get in the way. “Legislative scorekeeping rules sometimes prevent cost savings from a change to one policy area to count as cost savings when they affect programs in another area, even when the evidence shows savings are achieved,” the report said.

Ironically, the report states that “while there is growing demand for evidence-based policymaking and calls for more information to be interjected into congressional processes, the evidence that the increased use of data improves programs is still being developed.”

The Bipartisan Policy Center, which inherited the congressionally mandated commission task of fleshing out evidence-based policymaking, said public opinion polls offer some hope for the effort. Over the last three decades, polls “suggest an overwhelming lack of trust in government, regardless of which political party is in control, the report said. But “Republicans and Democrats generally agree that government effectively responds to natural disasters, to setting workplace standards, and to protecting the environment.”

The parties also appear to agree that government does not work well in “breaking the cycle of poverty or at managing the immigration system,” the analysts noted. But “there are many policy areas where most Republicans and Democrats disagree on how to measure or interpret performance and effectiveness, ranging from approaches to ensuring healthcare access to strategies for strengthening the economy.”

Part II of the center’s report is due in April.