Watchdog: Budget Cuts, Lack of Centralized Authority Hurting Program Evaluation

Agency efforts to incorporate more evidence in program evaluation are uneven, in part because many lack a central authority and in part because budget cuts have crimped attendance at relevant professional conferences, a congressional watchdog found.

In a survey of performance improvement officers implementing evidence-based program decision making to comply with the 2010 Government Performance and Results Modernization Act, only 11 of 24 agencies reported committing resources by establishing a central office to evaluate programs, operations and projects (only half of which have stable funding), the Government Accountability office said.

Those agencies that did centralize reported greater use of evidence in decision-making, said the report requested by Sen. Tom Carper, D-Del., chairman of the Senate Homeland Security and Governmental Affairs Committee, and Rep. Elijah Cummings, D-Md., ranking member of the House Oversight and Government Reform Committee.

Half the agencies have stepped up efforts to improve evaluation capacity through hiring, training, conference participation and consulting experts, GAO found, but four to five reported declines in hiring and conference participation—the avenues considered most effective by survey respondents. Overall, about half the performance improvement officers reported increased use of evaluations as supporting evidence for management and policy decisions.

About half reported increases in staff participation in evaluation conferences and knowledge sharing forums, hiring staff with research and analysis expertise, training staff in research and evaluation skills, and consultation with external research and evaluation specialists.

A quarter of agencies reported having agency-wide policies or guidance concerning key issues in study design, evaluator independence and objectivity, report transparency, or implementing findings, the survey found. Two-thirds of the agencies reported evaluation coverage of less than half their performance goals.

 “Promoting information sharing through informal and formal evaluation professional networks offers promise for building agencies capacity to conduct evaluation in a constrained budget environment,” the auditors said. “Engaging program staff, regularly reviewing progress on agency priority goals, and holding goal leaders accountable can help build agency use of evaluation in decision making.”

GAO stressed the importance of public dissemination of program evaluation results—even if the practices don’t always influence decisions—to maintain accountability to Congress and taxpayers and win backing for program decisions.

Even so, only half the agencies reported congressional interest in or requests for program evaluation studies, and “few agencies reported consulting congressional and other external stakeholders in conducting their evaluation studies or developing their evaluation agendas,” GAO noted.

The report contained no recommendations, though the Office of Management and Budget reviewed it in draft.

(Image via larry1235 / Shutterstock.com)