Program performance measures aren’t living up to their potential, scholars say

Efforts aimed at improving how agency programs perform haven’t been as effective as they could be, according to a new report.

The study published by the Brookings Institution and conducted by Donald Moynihan and Stephane Lavertu looked at performance management reforms of 1993 and 2002 to help shape expectations and gain insight for implementation of a more recent 2010 law.

Past reforms, they found, did not affect federal managers’ administrative decisions or their allocation of money, although the changes did result in managers refining program goals.

Moynihan and Lavertu looked at data from surveys of federal managers using the 1993 Government Performance and Results Act and the Program Assessment Rating Tool, a 25-part questionnaire the George W. Bush administration used to rate specific federal programs on a scale ranging from “results not demonstrated” to “effective.” GPRA required managers to develop strategic plans and monitor agency performance. PART sought to replace what the Bush administration called an “ineffective tool” by refining the collection of performance data. The scholars concluded that both GRPA and PART had little direct effect on how performance data were actually used.

“This finding generally supports the claims of policymakers that these reforms did not fulfill their potential,” Moynihan and Lavertu wrote.

The authors said the performance review process was intended to foster what they called “purposeful use” of data for tasks such as adopting new administrative processes and allocating resources. Instead, the data was used passively, they said, to refine existing program measures and set new performance goals.  

“Behavior that is easily observable can be directed, and as a result becomes the minimum required to demonstrate cooperation with reform implementation,” Moynihan and Lavertu said.

The study also noted the challenges of using data to gauge the progress and performance of specific programs. Interpreting key data measures depended largely on the team or person running a program, the authors said.

“Characteristics such as experience, knowledge, craft and ingenuity may make some workgroups and individuals better able to interpret data,” they wrote.

In January 2011, Obama signed into law the 2010 GRPA Modernization Act, which updated the 1993 law by adding chief operating officers and performance improvement officers to each agency. It also required agencies to submit reports to Congress quarterly, instead of annually.