Measuring Up

f the Government Performance Project's findings are any indication, managing federal agencies is as much art as science. And the same is true of rating agency management, we've found, after three years of experience. Government Executive joined the Alan K. Campbell Public Affairs Institute of Syracuse University's Maxwell School of Citizenship and Public Affairs in creating the project in 1996. It is a unique marriage of university and magazine, academic study and journalistic investigation, rigorous evaluation and contextual exploration. Last year we issued our first ratings of 15 agencies. As we end our second round of ratings, we are continuing to refine and improve our approach. Our hope is that systematically measuring and rating federal management performance will prompt agencies to manage better. That, in turn, should improve the odds that they will achieve results that will help dispel Americans' distrust and disappointment with government.

Breaking from common practice in journalism and politics these days, the project illuminates examples of good management as well as bad in a variety of agencies chosen for their significant interaction with the public. We hope through continuous, comprehensive scrutiny to give citizens and lawmakers information they can use to better evaluate agency performance. We also intend this report of management successes and failures as a learning tool for agencies throughout government.

Under a grant from The Pew Charitable Trusts, the project combines the results of a broad-ranging survey of each agency with in-depth reporting to create a rich portrait of management capacity and the environment within which it is exercised. The project team then rates each agency in five areas-financial management; information technology management; human resources management; managing for results and, where appropriate, capital management. The ratings are based on 22 criteria developed and refined during the course of the project by panels of academic experts and government practitioners, as well as the project staff. Ratings in each management area are averaged to reach an overall management grade for each agency.

The project now has rated 20 agencies and revisited five of those graded last year. This year, we gave first-time grades to the Coast Guard, Army Corps of Engineers, Veterans Benefits Administration, National Park Service and Office of Student Financial Assistance. We reassessed the IRS, Immigration and Naturalization Service, Occupational Safety and Health Administration, Patent and Trademark Office and Federal Aviation Administration.

We've studied agencies' responses to our survey. (Only the Army Corps of Engineers failed to complete the survey.) We've pored over thousands of pages of agency documents and, in most cases, received unprecedented access to interview hundreds of agency officials, managers and employees, as well as agency clients, critics, customers and overseers. We have spent hundred of hours developing, discussing and assigning grades. In the process, we have learned that, while management goals and obstacles vary from agency to agency, some things hold true across government:

• Most agencies are not doing nearly as well as they could be.
• Given the relatively long history and complexity of their management challenges, most agencies are doing far better than we might expect.
• Elected officials often act in ways that make it very difficult to effectively manage agencies.

We've found that, in general, federal managers do a surprisingly good job of accomplishing agency missions against difficult odds. Further, we've found that improving those odds is a long-term, incremental and complicated task tangled in a net of competing national and local interests; changing economic, social and demographic conditions; conflicting political priorities; and lack of national consensus on the appropriate limits of government action and accountability.

Balancing the Report Card

As we have rated agencies, we've tried to strike a balance between honestly portraying government's obvious shortcomings and not losing sight of its usually unheralded accomplishments. For example, despite receiving decidedly mediocre GPP grades this year, the Education Department's Office of Student Financial Assistance (SFA) last year awarded more than $50 billion in aid to 8.2 million students. Veterans Benefits Administration chief Joseph Thompson rated his agency a "D-minus" based on its internal "balanced scorecard" performance rating when he spoke with us in January, but the agency nevertheless will dispense more than $22 billion in compensation and pension funds this year. VBA also provides life insurance worth $481 billion to 5 million uniformed service members and retirees, has provided mortgages to 300,000 homeowners and, last year, found jobs for 10,000 rehabilitated disabled veterans.

The Army Corps of Engineers has a deferred maintenance backlog of $329 million on its assets, including hydropower plants, more than 500 dams, recreation sites, a fleet of watercraft and world-class research laboratories. But despite its repair problems, the Corps still guarantees navigation for 2.2 billion tons of commerce annually, is a lead agent building public schools in Los Angeles County, is restoring the Pentagon, manages more than 11 million acres of real estate, is cleaning up the Chesapeake Bay and New York Harbor, builds chemical weapons disposal facilities in Russia, provides one-quarter of the nation's hydropower and protects wetlands.

Though the Coast Guard staff is the same size as in 1967, the service in 1998 responded to 38,700 distress calls, saved 4,000 lives, spent more than 110,000 hours protecting U.S. fishing grounds, seized 82,623 pounds of cocaine and 31,390 pounds of marijuana, caught 3,648 illegal migrants, conducted more than 50,000 merchant vessel inspections and maintained 50,000 aids to navigation.

With just 23,000 employees, the National Park Service oversees a far-flung inventory of 378 units spread over 83 million acres in every state except Delaware, and as far away as the Northern Mariana Islands. Park service employees maintain and protect these areas while serving more than 285 million visitors annually.

Taking nothing away from these accomplishments, however, it's fair to say that in every case, these agencies and the 15 we graded last year could be doing better. Last year, SFA was forced to recall millions of paper financial aid applications because they were printed with mistakes. The organization faces a massive integration effort to consolidate a host of computer systems developed over the past 30 years and managed by eight separate contractors. VBA is struggling to improve a 64 percent accuracy rate in processing initial benefits claims. Running huge construction projects is at the core of the Army Corps of Engineers' mission, yet its managers have all but abandoned the agency's online project management system because it can't provide the data they need. The Coast Guard's own commandant calls the service a "dull knife" due to staffing, maintenance and training shortages. The Park Service has tremendous problems tracking its budget and maintenance needs because each park keeps its accounts differently.

Credit Where It's Due

A key challenge for the Government Performance Project comes in assigning management grades that take account of plans and new projects aimed at correcting such failings. In addition, the project staff must decide how to assess agency problems that originate elsewhere. Only recently, for example, did SFA gain control over its financial management systems, which had been miserably run by the Education Department. We graded agency management, in the first instance, against the project criteria, keeping in mind the range of performance among this year's and last year's agencies. But we also gave credit, or at least the benefit of the doubt, where agencies had new leadership or had crafted impressive plans to address management problems. For example, VBA gained some credit because its new chief, Joseph Thompson, has put in place promising intranet-based systems for evaluating agency performance and aiding employee development.

However, we will hold agencies accountable for the results of their plans and new programs. Last year, for example, we credited the IRS for its wide-ranging and energetic plans for reform under new Commissioner Charles Rossotti. But this year we reassessed the reforms, finding them less than meet the eye, and noting that IRS reorganization has brought a reduction in tax revenues and enforcement and has created an environment ripe for tax cheats. On the other hand, upon revisiting the Patent and Trademark Office, we found that new PTO Commissioner Q. Todd Dickinson has turned around the deplorable labor relations that marred the agency's report card last year.

We tempered this year's judgments of SFA in view of the fact that just over a year ago it became a performance-based organization with a new chief operating officer, Greg Woods, and new freedom from personnel and procurement rules. The changes are aimed at improving results and the agency already has adopted a new mission statement and simple, direct performance measures. SFA's dire computer problems and its parent agency's failing financial management might have earned it worse grades, but for the promise of its new organization and leadership. We'll wait, watch and reassess once the PBO has a chance to affect agency performance.

Because poor financial management has been a key stumbling block for many agencies, some also got credit for making real progress in getting a grip on their spending and how it relates to program results. PTO, for example, got good reviews for fielding one of the only fully operational activity-based costing systems in the federal government. Already, Dickinson has used it to adjust PTO fees to more closely match the cost of providing services. The system "allows us to get down into the specific costs of programs, to make the kind of informed decisions you have to make about not only whether to keep or retain or modify a particular program, but the impact that kind of modification would have on other programs," Dickinson says.

The Coast Guard is on the verge of similarly sophisticated decision-making as it installs its own costing system, an on-line version that soon will appear on the desktops of all field managers. Using activity-based costing, "I can see if there is wide variation in the hours people are spending on inspection and then decide how to smooth the variation in those hours by, say, developing a job aid to help people fill out data reports," says Capt. Charles Miller, commander of Coast Guard Activities Baltimore, one of two commands testing the system. "Activity-based costing and risk management help you put more metal on the target," adds Capt. Joseph Nimmich, chief of the Coast Guard's plans, policy and evaluation office.

The Meat of Management

Other aspects of financial management continue to stymie many agencies. In January, Senate Governmental Affairs Committee Chairman Fred Thompson, R-Tenn., released a report citing $221 billion in wasted government money, some of it dating to the 1980s.

The report, which compiled past findings by agencies' inspectors general and GAO, cited $90 billion in unpaid taxes and $12.6 billion in estimated Medicare overpayments. House Republicans set eliminating waste, fraud and abuse in federal programs as a top priority for the second session of the 106th Congress. But talk of financial problems is cheap, while solutions turn out to be tough and often expensive.

As members of Congress often observe, some of government's difficulties with financial management arise simply from poor bookkeeping. The National Park Service, for example, can't get an accurate picture of its maintenance needs because each park has its own way of documenting and justifying budgetary requirements and because the agency has lacked financial management skills on staff. Agency officials and legislators hope a business planning initiative launched in 1997 may yield improvements.

The Federal Aviation Administration continues to struggle with the financial management problems that we noted in its GPP grades last year. It joins the Coast Guard and Army Corps of Engineers in having problems accounting for property, plant and equipment that are so severe as to have prevented clean audit opinions for a number of years. Like grades and report cards in other areas, annual audits required by the 1994 Government Management Reform Act draw lots of agency attention when they point up problems. But for top-performing agencies, audits are just a means to a greater end. "Though we're fixated on our audit opinion, the real meat is managing better," says Coast Guard Chief Financial Officer William Campbell. "A clean opinion is a litmus test of whether you are managing by an accepted standard."

Tracking down and accounting for property and facilities not only means clean audits, it is a vital step in more efficiently using reduced resources to accomplish growing missions. "If you have no idea of the age of your assets, they may cost you more because you're not keeping them up and then you can't use them, or they may cost a fortune to keep running," Campbell says. "The Coast Guard owns 90 percent of the property in the Transportation Department's consolidated statement," he adds. "We've got a lot of old junk." Indeed, to account for lighthouse property it owns on Baker's Island, Mass., the Coast Guard had to hunt down a deed signed in 1796 by then-Massachusetts Gov. Samuel Adams. Both FAA and the Coast Guard have accounted for most of their property, which should help Transportation get a clean audit opinion for fiscal 1999.

Coping With Capital Assets

Though some agencies are getting better at accounting for their property, maintaining and replacing equipment and facilities remain time-consuming and onerous jobs. More than half the agencies rated so far by the Government Performance Project own little in the way of capital assets-expensive items with a lifespan of two years or more-besides their computer systems. But for those with significant capital inventories, including four of this year's five agencies, capital management is central to improving performance. Much of government's equipment and many of its buildings are old and expensive to keep in service, but Congress members are loath to permit closures of any visible symbols of the federal presence in their states and districts. In addition, legislators are almost as balky about fully funding the upkeep of new capital assets, let alone those agencies are forced to keep in operation past their predicted service lives.

The Army Corps of Engineers is trying to get legislators to attend to a maintenance backlog that tops $300 million by sending Congress photographs of the effects of deferred maintenance. The National Park Service-with 16,000 permanent structures; 8,000 miles of roads; 1,500 bridges and tunnels; 5,000 housing units; 1,500 water and waste systems; 200 radio systems; more than 400 dams and more than 200 solid waste operations-has estimated its repair backlog at $5 billion. But Congress has provided only about $125 million a year in upkeep funds, because of the agency's problems collecting accurate data from its parks' various accounting systems. Park service officials hope to allay legislators' concerns with improved data generated by a 2-year-old project management information system and a 4-year-old process for prioritizing construction projects.

The Coast Guard, meanwhile, operates an exhaustive process for managing its boats, planes and other assets. The service stopped merely replacing assets one-for-one as they wore out and instead created a new approach based on a painstaking study of the likely maritime environment of the future. An integrated plan for operating at sea, known as Deepwater, is a product of that process. It will upgrade equipment used for missions 50 miles or more offshore using a proposal for contractors that outlines not specific equipment, but the performance goals the Coast Guard believes it needs to meet in the future.

Using strategic planning, activity-based costing, risk assessment and capital programming, the Coast Guard hopes to overcome serious shortfalls in funding for the upkeep and replacement of equipment. But its extensive plans often founder in the shoals of politics, when presidential administrations and legislators add new missions without additional funds to accomplish them, add new equipment without appropriating money to staff and operate it, and refuse to allow the service to shift resources out of unneeded facilities. The Coast Guard isn't alone in facing expanding expectations along with static or contracting funding. While refusing additional spending on the Park Service's maintenance backlog, legislators add five new parks a year, on average, to the agency's inventory.

The 1993 Government Performance and Results Act, with its focus on matching appropriations to results, is designed, in part, to compel legislators to consider the performance costs of their decisions about what to fund and which programs to create or terminate. If it operated as designed, the law would offer some hope to agencies continually confounded in their attempts to more wisely manage programs and assets. But the law doesn't alter the appropriations process, the structure of appropriations committees or the way budgets are created. If only agencies, and not elected officials, are held to GPRA's performance standards, the law is unlikely to fully achieve its intended improvements in the conduct and results of government programs.

In January, GAO issued a report recommending ways agencies could improve their communications with Congress about performance (GGD-00-35). Among other things, the auditors noted that agencies had posted on their Web sites a good deal of the information sought by congressional staffers. The problem was that the agencies had failed to specifically tell the staffers the information was on the Web sites. This observation raises the question of just what sort of oversight is being exercised by a Congress whose staffers don't even monitor the Web sites of the agencies their committees oversee. Further, GAO found, Congress wanted reports that more clearly link agency resources, strategies and goals. But as the Centers for Disease Control told auditors, agency budgets often are structured differently from the way the organizations are structured, making such linkages difficult to portray. Congress could go a long ways toward clarifying the reports it receives by helping agencies restructure their budgets to match the way they actually operate.

Legislative staffers also complained that agencies failed to consult them when crafting measures for their GPRA performance plans. Of course, the law requires such consultation only about strategic plans, but agencies probably would save themselves some grief and extra reporting by including measures their congressional overseers are likely to request regularly.

Partnering and Collaborating

Most federal managers are too smart and too wary of statutory quick fixes to rely solely on management reform laws to improve operational efficiency and effectiveness. Agencies are hedging their bets by partnering and collaborating with their clients and customers and investing more time and energy in public relations. By doing so, the agencies hope to build and expand their bases of support, focus on results their customers really care about, and win backing for tough resource decisions.

Coast Guard Commandant James M. Loy, for example, has vastly increased the amount of time he spends testifying before Congress and making speeches in order to plead the service's case for better funding. Pitted against politically popular programs such as the Federal Highway Administration and the FAA inside the sprawling Transportation Department, the Coast Guard lacks a powerful advocate within the executive branch or Congress. It also lacks a strong, loud constituency lobbying for its needs. So the service is working harder at telling its story to win broad-based public support. Loy also is assiduously courting other federal departments, such as Justice and Defense, that benefit from the Coast Guard's work.

These efforts are bearing fruit. The congressional Coast Guard caucus has grown from three members to 60 and still is growing. A presidential commission that included seven Cabinet agencies and seven White House organizations unanimously approved the Coast Guard's approach to its deep-sea missions. The service's Deepwater acquisition has won interest and support from a broad range of Defense companies. "That is all a concerted, focused effort to raise the visibility of the value of our service and to solicit support so that when I go to the Hill to get the wherewithal to organize, train and equip my people, [the supporters] will be ready to sign up for that and make the difference in the appropriations process," Loy says.

The IRS, too, has learned the lesson of coalition-building. Almost every change it has undertaken started with extensive consultation with those who might be affected. Wisely, the IRS has included GAO representatives in meetings of the executive committee overseeing development of the agency's new tax processing systems. Inclusion in top-level decision-making means GAO is less likely to later demand more information. "GAO is an active part of this process, every step of the way," says IRS Chief Information Officer Paul J. Cosgrave. Including GAO is "a much healthier way of working" he says, and recommends other agencies try it. Thus far, it has brought the IRS smoother relations with the Hill, and, consequently, better treatment by the press.

SFA polls students, colleges and private lender institutions for ideas on improving service. Last year, a task force issued 200 recommendations for better service after compiling more than 8,000 comments and suggestions from customers and SFA employees. SFA is measuring customer satisfaction using the American Customer Satisfaction Index (ACSI), which scores organizations from 0 to 100 based on customer expectations, perceived quality and perceived value. In its first year participating in the ACSI, SFA received a score of 63 based on surveys of students who applied for financial aid. Students primarily complained about the timeliness and fairness of the application process. SFA chief Woods has set a goal of boosting the agency's rating to 74, the average for the private sector financial services industry, by 2002. Next year, Woods is going to measure 10 customer segments using the ACSI instead of just one.

The FAA collaborated with the airline industry on its Safer Skies program to reduce the most prevalent causes of airplane crashes. "We identified together where we could put our major emphasis and really focused on those areas," says FAA Administrator Jane Garvey. Another instance of cooperation occurred this summer when flight delays hit intolerable levels. High-level industry and FAA officials holed up together for two days generating recommendations for solving the problem.

The Corps of Engineers also is working to solicit and balance the often conflicting interests of those it serves. For example, twice a year, Maj. Gen. Phillip Anderson, commander of the Corps' Mississippi Valley Division, boards a towboat called the Mississippi and travels its namesake, stopping in countless ports to meet with scores of farmers, business owners, fishermen, environmentalists, barge operators, shippers, sportsmen, levee boards, community activists, politicians and anybody who has an interest in the river. He hears from crawfishermen, who want more fresh water diverted from the river to the marshes; shrimpers, who want less water diverted; and barge owners, who want the river kept high enough to ease navigation-but not too high. "It's a balancing act," says Anderson. "If everyone is equally displeased, you're probably doing the right thing."

GPP report card

Return to GPP home

Return to 2000 GPP issue