Trump Budget’s Blessing of Program Evidence Could Cut Two Ways

The Trump administration’s fiscal 2018 budget released on Tuesday couples calls for sizable spending cuts with language encouraging agencies to engage in evidence-based evaluation of programs.

That discipline—embraced by the Obama administration and the subject of a new statutorily created  study commission—has bipartisan backing. It is designed to rely more heavily on continuous evaluation of objective data from multiple sources to measure a program’s effectiveness and outcomes, while relying less on “input” measures such as spending levels and numbers of people participating in a program.  

But some specialists fear that when applied alongside a shrink-the-government political agenda, the process might lead to distortions.

In a section titled “Building and Using Evidence to Improve Government Effectiveness,” the Trump budget expanded on a shorter version released in March and recommended that centralized agency evaluation offices play an “impor­tant role in an evidence infrastructure that can develop and sustain agency capacity to build and use evidence.”

It noted that a recent Government Accountability Office report found that “federal agencies with a centralized evaluation authority reported greater evaluation coverage of their performance goals and were more likely to use evaluation results in decision making.”

One example provided was the Education Department’s “signature tiered evidence program” called the Education Innovation and Research grant program for “private school choice.” Trump is requesting $370 million for that program, with $250 million reserved for building evidence. As another example, the administration “is requesting that Congress give the government’s disability programs authority to mandate participation in demonstration projects. With this author­ity the administration proposes to conduct an aggressive set of rigorous experiments to improve the labor force par­ticipation of people with disabilities.”

Overall, the budget encouraged agencies to adopt a “learning agenda, in which they collaboratively identify the critical ques­tions that, when answered, will help their programs to be more effective, and to plan to answer those questions using the most appropriate tools.” That should produce “questions that re­flect the priorities and needs of administration and agency leadership, policy and program offices, pro­gram partners at state and local levels, researchers” and other practitioners, the budget said.

The Trump team’s effort to build on the movement for evidence-based policy “is worth celebrating,” said Robert Shea, an Office of Management and Budget performance specialist during the George W. Bush administration and now a principal for public sector at Grant Thornton LLP.  “To ensure we build on past efforts to use evidence in our oversight, management, and budgeting of programs will take the concerted efforts of officials across all agencies and branches of government,” he told Government Executive.

Shelley Metzenbaum, a data and performance official at OMB under Obama, agreed that the Trump team’s sentiment is “sound. The question is: how will this administration translate those words into practice?” she wrote in an email. “Will agencies be supported in their efforts not just to strengthen their and their delivery partners' measurement, analytic, and evaluation capacity, but also to use data to set priorities, find ‘bright spots,’ test hypotheses, test to see if what works well in one place can be replicated in others, and promote broader adoption of increasingly effective and cost-effective practices? Will agencies be able and encouraged to share current and historic data, trends, and evaluation findings in open and easily accessed ways, or will efforts to generate the data needed for strong evidence-based decisions be squashed?”

Metzenbaum wondered whether the administration will ignore “robust evidence” that, for example, boosting Internal Revenue Service taxpayer compliance efforts can make the tax system fairer. “My fear is that claims of evidence without supporting credible documentation will be used primarily to justify program cuts, not to improve the effectiveness, efficiency, and understanding of government operations,” she said.

The thrust of Trump’s budget, said Patrick Lester, director of the nonprofit Social Innovation Research Center in Annapolis, Md., who writes frequently on program evidence for Government Executive, appears to be “drawing on evidence to justify cuts.” He said he would like to see a debate over budget increases and decreases in which evidence plays a role. “But rather than seeing evidence drive policy making in a nice and rational way, we are in danger of letting evidence be driven by politics,” he added. 

“At the end of the day, politics is about politics, and it’s not surprising that the administration might be cherry-picking evidence to justify its proposed cuts” in such programs as Medicaid, teacher quality, college aid and international food aid, Lester said. He added that narrower legislation advancing evidence-based reviews often passes quietly if no one makes a political issue of it.

He said he expects the administration to continue its consideration of reinstating the George W. Bush administration’s Program Assessment Rating Tool.

The Trump budget’s call for agencies to recommend efficiencies comes at a time when agencies are designating new program management officers under a law signed by President Obama late last year and designed, not simply to cut, but to reduce waste and improve performance.

One of that bill’s key backers, Mark Langley, president and CEO of the nonprofit Project Management Institute, told Government Executive:  “To the extent the administration seeks to promote the critical assessment of programs before they are formally undertaken to anticipate challenges, identify solutions and clearly establish their strategic value, we favor the effort. If the evidence-based approach is designed and implemented in a way that builds upon strong implementation of legislation such as the Program Management Improvement and Accountability Act, we believe Americans will benefit from more efficient investment of their tax dollars.”