Performance Data: The Next Steps in Moneyball Government

Hope springs eternal. Two recent reports, a new book and a newly introduced bill in Congress all contribute to a steady momentum toward evidence-based decision-making. The legislation, which is bipartisan and bicameral, could have some momentum in the waning days of a lame duck Congress. It would create a commission to strengthen the use of evidence in policymaking.

In mid-November, the Government Accountability Office released an assessment of how well agencies use performance information in decision-making, with a particular focus on program evaluations. Around the same time, a study on evidence-based policymaking in the states was released by the Pew Charitable Trusts and the MacArthur Foundation; and a new book, Moneyball for Government, by Peter Orzag and Jim Nussle, was announced by the nonprofit Results for America. Finally, a bill was introduced in the House by Rep. Paul Ryan, R-Wis., with the support of Sen. Patty Murray, D-Wash., which would create a commission to recommend ways agencies could better integrate and use existing data, and use program evaluation techniques, to improve decision-making.

The GAO Report

GAO set out to assess the “extent of agency evaluation capacity” and the progress agencies have made over the past four years, since the GPRA Modernization Act was passed, in improving that capacity. It also looked for ways agencies have built their evaluation capacity, given that a 2013 GAO survey found that only about a third of program managers reported recent evaluations of their programs.

GAO surveyed the performance improvement officers from the 24 largest agencies, and their capacity to conduct and use program evaluation was uneven: “seven of the 24 agencies have central leaders responsible for evaluation; in contrast, seven agencies reported having no recent evaluations for any of their performance goals.”

GAO observed that an agency’s evaluation capacity depends on both the demand for such information by policymakers, as well as the agency’s ability to supply credible and relevant information. The report identified three elements in effective agencies:

• An enabling environment where there is interest in evidence of government performance

• Organizational resources to support the supply and use of credible evaluations

• Evaluation results and use linked to an agency’s management, policymaking and budgeting processes

An important thread throughout the report was the observation that it wasn’t the supply of performance information that made a difference in its use, but rather the analysis and interpretation of the data for leaders.

In addition, GAO found that only about one-quarter of the agencies in its survey “reported that their evaluation offices had a stable source of funding” and concluded that congressional committees could “communicate their interest in evaluation” in a number of ways. GAO made no specific recommendations, however. Interestingly, President Obama recently signed the reauthorization of the Child Care and Development Block Grant, which includes a half percent set-aside of funding for program evaluation, demonstrating continued congressional interest.

The Pew-MacArthur Study

This succinct guide focuses on what state governments are doing to assimilate performance and evaluation information into decision-making. It found that over the past decade, 100 state laws across 42 states have been adopted to support the use of evidence-based programs and practices. Based on these and other state activities, the guide offers “a framework that governments can follow to build and support a system of evidence-based policymaking.” Its framework has five components:

• Program assessment. “Government leaders should develop an inventory of programs they currently operate and then assess the available evidence of effectiveness and return on investment for each one.” For example, Rhode Island developed “an inventory of 58 state-funded programs intended to reduce recidivism in adult and juvenile justice systems.” It found only one-third were not evidence-based and plans to improve its efforts.

• Budget development. “Executive branch agencies should use performance information when developing their budgets” to ensure funding is directed to programs that work. For example, Connecticut examined its investment in early reading programs and found there was no positive effect on reading skills. In digging into the data, they found the monies were actually being diverted to other purposes.

• Implementation oversight. “Even the most effectively designed interventions can produce poor results when poorly run,” so the guide recommends strong monitoring systems. For example, a number of state governments and agencies are using regularly scheduled PerformanceStat forums to identify areas for improvement.

• Outcome monitoring. “Routinely measure and report outcome data to determine whether programs are achieving desired results.” For example, “Virginia Performs is an interactive, publicly available database that collects and reports performance data on a wide range of government functions at multiple levels” (program, agency, region, etc.).

• Targeted evaluation. “Conduct rigorous evaluations of new and untested programs to ensure they warrant continued funding.” For example, a new “probation with enforcement” program in Hawaii combined data from numerous sources to assess its effectiveness. By linking data from drug test results, offenses, supervision activities, etc., the program was found to have reduced rearrests by 55 percent.

The guide notes that several federal grant programs have provided incentives for states to invest in evaluation because they targeted about $5.5 billion for seven initiatives that support proven programs.  For example, the federally funded Maternal and Infant Early Childhood Home Visiting Program “requires grantees to direct 75 percent of federal dollars to evidence-based programs and evaluate the impact on key outcomes.”  

Moneyball for Government

Released in mid-November, this book is a compilation of essays by a high-level bipartisan group of advocates for evidence-based decision-making, under the aegis of the Results for America advocacy initiative. According to the press release:

 “This book is about addressing our nation’s great challenges by getting better results with gov­ernment investments. By shifting public resources toward solutions that use data and evidence of impact, we can improve the lives of young people, their families and communities,” said Michele Jolin, managing partner of Results for America. “By bringing together leading thinkers from across the political spectrum and highlighting what is working across the country, this book of­fers a way to open up new opportunities and to make dramatic progress in improving lives. We call upon government leaders at all levels—as well as presidential candidates from both parties—to support and implement the bipartisan ideas in this book.”

Pending Legislation

In late November, a bill to create the Commission on Evidence-Based Policymaking was introduced by Murray and Ryan, the Senate and House Budget Committee chairs. Champions of the legislation hope that it would be attached to the “must pass” bill to fund the government after the continuing resolution expires Dec. 11.

According to the bill, the 15-member commission would offer recommendations on how administrative, tax and other data collected by the federal government “may be integrated and made available to facilitate program evaluation, policy-relevant research and cost-benefit analyses.” It would also assess the feasibility of creating a clearinghouse for agency program and survey data.

So, with all the recent activity on using increased evidence, it will be interesting to see whether the new Congress continues the momentum.

(Image via Africa Studio/Shutterstock.com)