Results--or Else

group of unlikely specialists is joining doctors and nurses attending auto accident victims in Washington-area hospitals these days-federal auto safety regulators.

The rulemakers want to know what results, negative or positive, may be flowing from the safety requirements the government has attempted to impose on American auto fleets. They're in the emergency room for a firsthand look at the carnage.

Engineers, managers and lawyers from the National Highway Traffic Safety Administration have been trooping to trauma centers during the last two years led by NHTSA Administrator Ricardo Martinez, who is himself a former ER physician. NHTSA manager Noble Bowie, for example, spent a 4 p.m.-to-midnight Friday shift last December at the University of Maryland shock trauma unit in Baltimore. Watching helicopters disgorge accident victims and emergency room staffers bind wounds helped Bowie understand Martinez' push to refocus NHTSA on injuries rather than just fatalities caused by the nation's 6.3 million reported crashes a year. "It's good because it orients you towards the bottom line and the results of what we do," Bowie says.

Results rule all over government right now as agencies scramble to meet their first deadlines under the 1993 Government Performance and Results Act (GPRA). Managers are humping to meet the Sept. 30 GPRA deadline for delivering agency strategic plans to Congress and the President. One of the toughest challenges they face is making clear the connection between agency programs and real world results.

GPRA is intended to improve confidence in government by holding agencies accountable for achieving results that affect taxpayers' lives. The law compels agencies to set outcome goals, measure their performance and report their accomplishments. It's also supposed to help Congress make better funding and oversight decisions by giving legislators credible information about agency efficiency and effectiveness. At fiscal year's end, all major agencies must deliver to OMB and Congress strategic plans including mission statements, outcome-based goals and an explanation of how the goals will be achieved and progress will be measured. Also this fall, agencies must deliver their first annual performance plans linking strategic goals and day-to-day activities. Ultimately, agencies will report each year on progress toward meeting their goals. The first GPRA performance reports are due March 31, 2000.

GPRA took effect with 70 performance planning and reporting pilot projects in agency programs that began in fiscal 1994 and ended in fiscal 1996. An OMB report on the projects was due May 1.

Though they've had four years since GPRA became law, agencies are finding preparing strategic plans much tougher than expected. That's in no small part due to the "damned if you do, damned if you don't" nature of the exercise. Managers have discovered that planning and measuring performance can turn operations upside down; sharpen disagreements with Congress about an agency's mission or its existence; and require predicting outcomes shaped by forces agencies don't fully control. Managers also have found, to their chagrin, that planning and measuring could lead to a program's demise if it isn't achieving results or the results achieved aren't deemed to be worth the cost.

What's more, Republican legislators-who have made no secret of their desire to deep-six a number of agencies-are gleefully embracing the long-ignored law. The GOP congressional leadership has set out a host of expectations for congressional consultation sessions on agencies' strategic plans. The House majority has created 25 cross-committee task forces covering major agencies. The task forces are rapidly educating congressional staffers about the law Republicans have dubbed the "Results Act." Staffers also are learning how best to grill agency officials about GPRA plans and performance measures. As Congress tightens its grip on GPRA, the Office of Management and Budget is exerting an equally strong pull for agency performance plans to incorporate into the annual budget. The GPRA tug of war has left managers feeling stretched.

Consultation Controversy

During a January retreat, the House GOP picked Majority Leader Dick Armey, R-Texas, to run implementation of the Results Act and enlisted Budget Committee Chairman John Kasich, R-Ohio, Appropriations Committee Chairman Bob Livingston, R-La., and chief of the Government Reform and Oversight Committee, Rep. Dan Burton, R-Ind., as Armey's lieutenants. House and Senate Republican leaders threw down the GPRA gauntlet Feb. 25 with a letter to OMB Director Franklin Raines announcing their intention to play a major role in the law via congressional consultations during strategic planning. The letter included a long list of issues the GOP wants addressed by agencies during consultations and a warning to OMB not to delay the confabs by forcing agencies to vet their plans before meeting with legislators.

Raines responded March 17, promising OMB review wouldn't slow consultations and reminding Republicans their proposals will be heard, but they won't all be incorporated in agency plans. "Ultimately these are the agencies' and the President's plans," says John Koskinen, OMB associate director for management. "Where there are disagreements, we will acknowledge them and say, 'Here's where we're going and how we will measure effectiveness.' If Congress wants to change that, and they can override the President's veto, they can change the [agency's] statute."

Observers say Republicans are barely disguising their intention to use GPRA performance data as ammunition in their battle to reduce the number of agencies. GOP legislators apparently feel certain that, held up to the glare of results, many programs can't cut the mustard. Virginia Thomas, Armey's committee liaison, concedes as much, but argues that this is exactly the purpose of GPRA. In attempting to close agencies at the start of the 104th Congress, "Republicans made some errors and tried to move too fast," she says. "They didn't have credible information to back up their legislative reforms and they didn't get the public behind them." But now, she says, when budgets are declining and a Democratic President has declared the era of big government over, "whatever changes will be made and whatever budget decisions will be made should be made on the basis of credible information. There will be information from the Results Act that's detrimental to some agencies. There's no way to avoid that.

"We're not at the point where we're sharpening knives or setting the sights on bomb targets. It's not a hostile environment; we're working with OMB," Thomas says by way of consolation for managers dreading their congressional consultations. Indeed, OMB's Koskinen encourages agencies to step up to the Republican challenge. "This process can demonstrate government does some things very well. If the challenge is that [Republicans] do not think agencies can do it, agencies ought to be excited to meet that challenge." In fact, GPRA gives federal managers a chance to shine, says Christopher Mihm, General Accounting Office assistant manager for federal management issues. "What GPRA allows is for managers and agencies to put their best foot forward. [It] positions you to be viable, and in the current environment, that's the best you can hope for. The alternative is that decisions will be made about your program without any performance data."

GPRA Grades

By early spring, fewer than half the largest agencies had contacted congressional committees about their strategic plans. And those consultations fell far short of the mark, according to the GOP staff task forces. "It was disappointing in terms of the level of the consultations and the level of paper flow moving between the branches. No agency got over a C, most got Incomplete, C- or F from the congressional staff based on what we expected to see at this point," says Thomas. Staffers say agencies are briefing them at too general a level, failing to differentiate between outputs and outcomes in their strategic plans, and confusing tools-such as disseminating information-with goals-the reason information needs to be disseminated.

Despite the Results Act fanfare, Congress has had its own problems gearing up for consultations, Koskinen says. "Some agencies are involved in vigorous consultation. Others haven't heard from their committees. Some agencies have received detailed questions from their oversight committees or appropriators. Some have been told, 'We don't want anybody to think it's our plan so we're not co-opted and we can raise questions later.' Some have heard nothing." Spotty consultation performance worries Mihm, who believes the meetings are good indicators of GPRA success. "Consultation isn't preparing for GPRA, it's doing GPRA-asking what are our priorities, what are we doing, and what are we willing to have go away if we don't have the money? By June, if we're not seeing a lot of effort by agencies and Congress at engaging, then the consultation opportunity would have been lost," Mihm says.

Some Congress members and staffers have a firmer grasp on the law and its intent than others. For example, officials from a GPRA pilot agency say that when they reported setting a goal of 50 percent improvement in an outcome and achieving 25 percent, appropriations staffers asked, "You only achieved half your goal, why shouldn't we cut your budget?" Staffers like these "have not learned the lesson of the Results Act," says Paul Rosenzweig, investigative counsel with the House Transportation and Infrastructure Committee. "The whole point is to move toward rewarding good performance, in other words, the partial achievement of good goals." Those with the best grounding in GPRA understand that it implies a cultural and managerial revolution in government, says Rosenzweig, who leads the Transportation Department Results Act implementation task force. "It's a very hard thing to reinvent yourself this way," he says. As a result, legislators may need to ratchet back their GPRA expectations, at least at first. "In Congress, the risk is that people will feel the strategic plans are final and if there are changes, that will mean agencies didn't do the first ones correctly. If they never change, they probably were not the right plans. The nature of the process is that it goes on forever," Koskinen says.

Stellar Planners

One measure of GPRA success may be the pain the law is causing. "We are finding that the agencies making the most progress in implementing GPRA are those that recognize they still have many implementation challenges to solve, while those making the least progress tend to see little difference between the requirements of GPRA and the way they have traditionally done business," Acting Comptroller General James Hinchman told the House Government Reform and Oversight Committee Feb. 12. The folks most familiar with GPRA challenges are those involved in the 70 performance measurement pilot projects that began in 1994. Two Transportation Department components, the Coast Guard and NHTSA, were among them and offer lessons about the trials, tribulations and payoffs of good strategic planning.

NHTSA, an agency of approximately 650 employees with a 1996 enacted budget of $277 million, works to reduce the nation's 40,000 traffic fatalities and 3.1 million injuries each year. The societal cost of these accidents is estimated at $138 billion a year. NHTSA's mission statement says it will "save lives, prevent injuries and reduce traffic-related health care and other economic costs."

Use the Right Data

The agency became a GPRA pilot with some distinct advantages. NHTSA had begun a strategic planning process in 1992, before GPRA was enacted. In addition, NHTSA was blessed with an extensive, decades-old data-gathering system that allowed the agency to measure the outcomes of its activities well before that became fashionable. The data system gave NHTSA a good idea of the impact its programs were having, a key GPRA indicator.

"We had been monitoring important measures and establishing goals for seat belt usage, the fatality rate, and percentage of accidents that were alcohol-related," says Bowie. Another important plus is that responsibility for the GPRA pilot, National Performance Review efforts, the agency budget, and strategic planning all rest in the same NHTSA office, says Bowie.

NHTSA's data advantage is underscored by the Coast Guard's problems in this area. The Coast Guard GPRA pilot encompasses the Marine Safety and Security (MSS) and Marine Environmental Protection (MEP) programs, with staffing of about 3,400 and annual spending of $460 million. Progress toward the Coast Guard's goal of reducing accidental maritime deaths and injuries is measured by monitoring the number of deaths per 100,000 maritime workers. But the measure relies on Bureau of Labor Statistics data on the number of maritime workers and that figure has fluctuated dramatically year to year. To improve data quality, the Coast Guard indexes BLS data using a 1987 National Marine Fisheries Service survey and has begun using multiyear rather than annual data.

Both the Coast Guard and NHTSA pilots emphasize the importance of collecting the right data and making sure it is accurate. "We had a data culture with tons of information you can't make any sense of," says Coast Guard Commander Christopher Boegel, chief of strategic planning and analysis for the GPRA pilot. "Information which is valid, clear, comprehensible and comparable to some basis, is different in terms of the equipment and people you need. Information implies quality and that implies time and effort." Boegel says the lessons learned by the GPRA pilot are being applied as the Coast Guard develops a new marine safety information system.

Link Outcomes to Activities

Bowie attributes NHTSA's success at linking its activities to outcomes to more than just data. Program evaluation plays a role, too, he says. All regulations proposed by NHTSA must undergo cost-benefit evaluation. A post-evaluation group revisits rules several years after they take effect to confirm they're performing as expected. GPRA will produce a boom in program evaluation, predicts Coast Guard Cmdr. Jean Butler, who is on detail as a member of Transportation's strategic planning team. "Typically evaluation is not something in government that's a big priority." Traditionally, agencies begin programs because laws require them, Butler says. Agencies rarely evaluate the results of those programs. "You just start a program and move on," she says.

Evaluating their emergency room experience helped set NHTSA staffers on the road to crafting new auto safety rules, Bowie says. Doctors showed their NHTSA visitors hundreds of X-rays of knee, ankle and foot injuries caused by high-speed auto accidents. Severe lower extremity injuries are common in such crashes, but they used to be beside the point because hardly anyone survived. Now that NHTSA requires air bags in new cars, people are surviving the crashes but suffering debilitating leg injuries. Under Martinez, NHTSA has added a campaign to prevent injuries to its traditional focus on reducing traffic fatalities. That campaign, along with Martinez' ER field trips, has given birth to a new regulation to prevent lower extremity injuries by improving frontal crash protection in automobiles.

NHTSA also is a trailblazer in involving stakeholders in developing strategic plans. In 1992, it published in the Federal Register some 30 questions about the future of highway safety, technology and how NHTSA does business. More than 100 responses came back from stakeholders including state and local governments, the auto and insurance industries, other federal agencies and advocacy and consumer groups like Mothers Against Drunk Driving and Public Citizen.

In addition, NHTSA and the Coast Guard are ahead on identifying the external factors likely to affect outcomes. For example, NHTSA factors into its outcome planning the fact that traffic fatalities rise and fall with the economy as people drive more at night for entertainment during flush times.

Likewise, when the Coast Guard discovered marine towing industry accidents were caused mostly by human error, it moved resources away from inspecting vessels toward teaming with the industry to develop voluntary guidelines and training. Towing fatalities fell from 91 per 100,000 employees in 1990 to 27 per 100,000 in 1995.

The GPRA experience of NHTSA and the Coast Guard is hopeful for other agencies. "In terms of our budget, it has served us well," says NHTSA's Bowie. "Other agencies were getting cut while NHTSA was experiencing growth in the fiscal 1996, 1997 and 1998 budgets. If you looked at other regulatory agencies a lot were under fire and targeted with severe budget cuts and we came through unscathed. We strengthened our case by saying for X amount of money, we'll give you this outcome." The Coast Guard GPRA pilot rescinded or relaxed all internal specifications not required by law or regulation for how work was done in marine safety programs. That let managers and commanding officers in the field substitute their own innovative practices and save hundreds of thousands of hours of work each year. "Before we go asking Congress for more flexibility, we're making sure we're flexible ourselves," Boegel says.