Bush performance rating tool draws plaudits, pans

As the Bush presidency winds down, performance reviews of the administration's Program Assessment Rating Tool are mixed.

Proponents of PART, which uses a standard questionnaire to assess and improve every federal program, laud the tool for providing a wealth of new performance data, while critics argue that it fails to recognize the diversity of individual programs, among other things.

The two groups made their respective cases in Washington on Wednesday and Thursday.

Beryl Radin, a professor at American University, argued at the American Bar Association's Administrative Law Conference on Thursday that PART and the 1993 Government Performance and Results Act were too narrow in scope to meet their goals.

Both concepts, she said, focused on holding federal employees accountable for outcomes and results, but did not incorporate the cause of those outcomes -- such as air quality or the number of drunk driving fatalities -- into their analyses.

"Certain outcomes are difficult to measure," Radin said. "And this has the effect of blaming the bureaucrat."

The difficulty of measuring how well programs and individual agencies are meeting their statutory missions is not new.

Sid Shapiro, a law professor at Wake Forest University, said GPRA -- passed during the Clinton administration -- has failed to promote effective regulatory government because its inherent goal is to ferret out waste, fraud and abuse and then punish underperforming agencies by cutting their budgets. Consequently, agencies attempt to protect themselves by devising "euphemistic performance goals in order to assure that they can pass their own grading criteria," Shapiro wrote in an essay presented at the forum.

Annual GPRA reports, he said, rarely mention the agency's lack of funding or staff reductions, and how those challenges may affect their performance.

"These reports are not really dealing with the realities facing these agencies," Shapiro said. "GPRA occurs outside any real accountability."

Shapiro and Radin are working on a book about the "deterioration and dysfunction" of federal health and safety agencies and how the creation of positive metrics could attract more oversight and publicity to the causes of their regulatory problems.

To improve the quality and use of federal performance data, Radin suggested that the next administration recognize that a one-size-fits-all approach will not work and assessments must suit the unique goals of each program.

She also recommended that the Office of Management and Budget take a less "command and control" role in the process; congressional appropriators become more involved in using the data for funding decisions, and the White House provide better interagency coordination to share best practices.

The skepticism over PART's effectiveness and the state of federal performance management is not shared by those involved in the effort during the Bush administration.

Robert Shea, OMB's former associate director of administration and government performance and one of the architects of PART, addressed the merits and relative successes of the tool during a virtual forum on Wednesday hosted by Cognos, an IBM company.

After more than six years, PART now has evaluated every federal program -- more than 1,000 in total -- and suggested specific management, legislative or regulatory improvements.

When PART began in 2002, 50 percent of all federal programs evaluated could not demonstrate their results and only 6 percent were rated "effective." Now, nearly 50 percent of all programs are rated as "effective" or "moderately effective" while less than 20 percent are ranked as "results not demonstrated."

The focus of these evaluations also has evolved, Shea said. For example, until recently small business development centers only measured the number of small businesses they counseled or trained. Now the centers look at the number of jobs created. Likewise, community health centers previously measured how many people they provided service to; now the target is health outcomes such as low birth weight in babies.

"More and more, programs are meeting their goals and measuring success today better than they were five or seven years ago," said Shea, who is now with consulting firm Grant Thornton's global public sector group.

Ted Kniker, an executive consultant with the Federal Consulting Group, a franchise operation of the Treasury Department, has been on both sides of the PART assessment. When Kniker worked at the State Department's Bureau of Educational and Cultural Affairs, his office was assessed several times by the PART; he now advises other agencies on improving their performance.

When ECA first was evaluated in 2004, the bulk of its programs received a grade of "results not demonstrated" because its measures were not linked to long-term goals and lacked a clear strategy.

Kniker and his colleagues reviewed and refined the measures, hired a performance measurement expert and linked PART to their planning process. ECA now is ranked as 92 percent effective and serves as model for the rest of State, he said.

"We had an internal culture change from the normal government processing -- 'move the money out the door and make sure you spend it all before the end of the fiscal year' -- to a focus on results and what it was that we were trying to achieve," Kniker said during the Cognos forum. "And PART prompted real discussion … about what those results were supposed to be and what those results could be."