On the Brink

The success of the 2010 census hangs on a risky switch to handheld computers.

The risky idea for changing the way the Census Bureau gathers the most consequential data about every American sprang from the mind of one man: Jay Waite.

In 2001, shortly after the bureau had cleaned up after the 2000 census, Waite, then head of the decennial census program, was convinced the bureau had to alter the way it conducted the 2010 count. The cost of the 2000 census had topped $6.5 billion, doubling from 1990, after adjusting for inflation. Initial estimates predicted the 2010 count would nearly double that, to $11.3 billion. The 2000 census also had come under sharp political criticism, with Democrats, local government officials and minority groups claiming it had undercounted minorities and low-income families.

It didn't take long for Waite to come up with an answer: use handheld computers to replace the paper-and-pencil method of collecting data from Americans who have not sent in their forms and move the costly and cumbersome long form, which has more than 40 questions, to another census survey program. "This idea, as it were, of going to automation and handhelds came out of my brain," he says.

He surprised himself. Waite describes himself as a farm boy and Luddite. But he says his transformation occurred during the 2000 census as he heard "more and more talk about cell phones and handheld devices. We tied our horse to that fact, and I bet that the technology would come to me."

Six years later, the bureau is poised to give handheld computers to the 525,000 enumerators it will hire to go door-to-door collecting information from the estimated 108 million Americans who will fail to send in 2010 census forms. The plan represents "the biggest move in a decennial census in our lifetimes," boasts Waite, who was promoted this year to deputy director, the No. 2 position at the bureau. "It is a huge change."

Waite's bet on technology to support a huge business-process change sends shivers down the spines of risk management experts. With so much riding on the outcome of the census (the data is used to reapportion seats in the House of Representatives and state legislatures, to plan state roads and highways, to build schools, to market goods and open businesses, among other things), experts say the decision to move to handhelds should have been supported by a full-scale analysis called enterprise risk management.

The Office of Management and Budget and the Government Accountability Office have strongly advised agencies to include risk management as a key element when undertaking new projects-especially huge ones. Enterprise risk management is relatively new. It expands the theories of project risk management to the entire enterprise, analyzing how a change or investment in a new technology might affect each business unit. It starts out, however, by questioning the reason for change in the first place. "The fundamental question to ask is not 'What's the problem you want to fix?' It's 'What do you want to accomplish?' " says Bill Sharon, chief executive officer of the consulting firm Strategic Operational Risk Management Solutions. "We are all very adept at describing what is wrong, but it takes a greater effort to figure out what is it that we want to go right."

That's where the Census Bureau has fallen short, a failure that cascades risk throughout the bureau, experts say. The decision to use handhelds is a classic case of an agency failing to incorporate enterprise risk management. As a result, the bureau has put the census at a much higher risk for failure. Already, problems are cropping up, for example, a five- to tenfold increase in the handheld contract requirements, cost overruns and lack of an overall recovery plan should the handhelds suffer interruptions of service or, worse, fail altogether.

"All this borders on professional IT malpractice-if not malfeasance," says Robert Charette, director of the enterprise risk management and governance practice of the Cutter Consortium in Arlington, Mass. "The handheld project is an IT blunder-not a failure-in the making. A failure is when you do all the things you should, but things still go off the rails. A blunder is when you don't do the things you know you should, many times because of hubris, and things go off the rails, predictably," says Charette.

Unintended Consequences

Because of congressional attention to project costs, when agencies consider risk, they tend to focus on potential budget overruns and deadline slippage. For that reason, agencies work hard to manage requirements risks, especially for information technology projects. They home in on only those requirements designed to meet performance goals for systems. Recently, security risks also have appeared on agency radar screens, thanks to media attention to identity theft from federal IT systems.

Such a narrow focus would not fly in the private sector, whose regulatory organizations have recognized the importance of enterprise risk management. The Basel II Accord, a set of guidelines for measuring bank risks put together by the Group of 10 countries, includes operational risks to the enterprise. As of the end of last year, the accord required financial services companies to carry enough capital to offset the companywide level of risk, which includes IT systems. The Treadway Commission's Committee of Sponsoring Organizations, a private sector organization formed in 1985 to combat fraudulent financial reporting, has an enterprise risk management framework that includes IT risks. And the Control Objective for Information and Related Technology also advises organizations to set up an enterprise risk management framework.

That means executives should manage risk across the enterprise, not just in silos. Changes to processes in the human resources department can affect, unintentionally, the business processes in the finance department. Likewise, changes to, say, business processes in the IT department can unintentionally affect manufacturing and thus the quality of the company's products. In the case of the Census Bureau, deploying technology to change the way the census is conducted can have numerous unintended consequences across the operation. To begin analyzing those risks, management experts suggest that organizations first must ask whether the investment in the change improves the organization's ability to meet its mission.

The agency should ask, "What do you want to accomplish?" says Sharon.

But that's not what happened at the Census Bureau.

A Solution Looking For a Problem

The bureau first formally presented its argument for using handhelds in June 2001, in a 10-page report titled "Potential Life Cycle Savings for the 2010 Census." Compiled by top managers, it lays out the plan to use handheld devices linked to the Global Positioning System to collect more accurate data on new housing units-GPS signals would provide better location information. The plan also called for administering the long form during the annual American Community Survey rather than the decennial census, to boost response rates and to reduce enumerators' workload. Using handhelds would eliminate the cost of printing, storing and moving the 20 million maps and 400 million census forms the bureau used in 2000. These costs soak up about two-thirds of the total cost of the decennial census, or about $4.3 billion of the $6.5 billion total in 2000.

The handhelds also would reduce the number of households that enumerators would need to visit, according to the report. In 2000, 4 percent of households visited already had sent in forms, but too late for the bureau to inform enumerators. Handhelds would be updated throughout the day about forms received, saving the bureau about $135 million, according to GAO estimates. Other savings would come from reduced mileage costs (5 percent per case) and increased productivity (5 percent). In all, the Census Bureau calculated it would save $2.9 billion by using handhelds instead of the pencil-and-paper method used in 2000. The bureau estimated the cost of the redesign to be $2.4 billion, bringing net savings to $445 million.

The report was presented to the National Academies of Science; top executives at the bureau's parent organization, the Commerce Department; and GAO. Waite says they all supported the bureau's approach.

But the report fails to address how moving to handhelds will improve the mission performance, says risk management expert Sharon. "From a risk perspective, you don't start with money," he says. "You start with the vision." The report gives passing reference to how the redesign will provide more accurate data-a key criticism leveled by advocacy groups and Congress during the 2000 census. "The 2010 census will be armed with a far more comprehensive, timely and accurate address list-one of the best predictors of a successful census-without the added complexity, risk, end-of-the-decade costs and last-minute address-building costs," according to the report.

But Sharon says a vision requires much more. It requires calling together operational managers from every line of business to advise on what needs to be accomplished and what must be done to accomplish it. In the census' case, what's the best way to count every American, as required by the Constitution, while avoiding the political fallout the bureau suffered in 2000? Only after developing several options should an organization begin pricing them. If the best option is a budget buster, managers must move to No. 2 or No. 3, Sharon says.

Other options could have included relying on the Internet, telephones or some other approach that might not use technology, such as better marketing techniques. (The Census Bureau says that years after issuing the life-cycle savings report, it tested the use of the Internet, but concluded that the online approach didn't appreciably increase the response rate. Census thus contended that the cost of developing an Internet program would be too high. As of last year, Congress insisted that the Census Bureau should be pursuing an Internet option using the IRS' electronic tax filing program as a model.)

'Handhelds Are the Keystone'

Armed with a document outlining $445 million in savings from adopting handhelds for 2010, the bureau began figuring out how to obtain the devices. After giving up on the idea that its IT shop could modify a commercial personal digital assistant, the bureau issued in January 2005 a request for industry bids. In March 2006, the bureau awarded the $600 million Field Data Collection Automation contract to develop and manufacture the devices to Harris Corp., a telecommunications company based in Melbourne, Fla., with experience supplying the Defense Department and law enforcement agencies with tactical radios. Losing bidders included Northrop Grumman and General Dynamics.

But just weeks before the award, GAO criticized the bureau's risk management strategy. In March 2006 testimony before the House Subcommittee on Federalism and the Census, David Powner, director of information technology management issues at GAO, said the bureau had identified some high-level risks and drafted a risk mitigation strategy. But, he added, "the FDCA project office has not yet revisited or analyzed the identified risks, begun prioritizing and tracking project risks, or documented risk mitigation plans. Until the team implements an effective risk management process, it will lack a mechanism to address known and unknown problems."

When Powner audits an agency's risk management strategy, he reviews performance of five tasks:

• A list of all known risks. "The biggest red flag is if we don't see 20 or 25 risks listed," Powner says.
• A scale on which risks are ranked. The scale can be from low to medium to high risk, or from 1 to 5. It doesn't matter, just so they are ranked.
• Aggressive mitigation. The highest rated risks are given to owners to mitigate, with a specific time frame. "When we go into the FDCA program office, we want to see activity that shows they are all over these risks," Powner says.
• Informed top executives. Project managers should inform executives of the highest risks and what action is needed from leaders to mitigate them.
• Aggressive follow-up. The minutes from meetings of project team leaders and risk assessment boards should show that members discussed risks, mitigation strategies and what progress is being made. "We don't want to only see a plan; we want to see action that's documented," Powner says.

For Census, managing the risk of the handheld contract was paramount, auditors have said for years. "The handhelds are the keystone to the reengineered census," says Mathew Scire, GAO's director of strategic issues. "The arch isn't going to stand if the computers don't work."

FDCA project managers at the bureau and at Harris stress that a lot has been done in the past year. "Much of what the GAO has pointed out as a problem, we have fixed," says Ed Wagner, FDCA project manger at Census.

The bureau has created a review board that meets monthly to go over risks that project members have identified-mostly cost- and schedule-related. Harris-which teamed with Dell Computer, Accenture, Oracle and High Tech Computer Corp. on the handhelds-has formed a similar board tracking mostly technical issues, says Mike Murray, vice president of census programs for Harris. Each board ranks risks, and project managers inform top executives about the most pressing ones. The boards document each new risk and assign it to an owner, who is responsible for tracking mitigation.

Murray outlines the company's response to the data security risk. He says securing information stored and transmitted by the handhelds is No. 1, especially because private data would be held on the devices for a short period of time and transmitted over the cellular phone network. To mitigate that risk, Harris equipped each handheld with a biometric scanner, which enumerators must use to gain access to the GPS system, census questionnaire and stored data. If the reader fails to recognize an enumerator's fingerprint after three attempts, it shuts down for 15 minutes.

In addition, all stored data is encrypted. Data transmitted over the cellular network also is encrypted as it is en route over a virtual private network that uses the cellular network as a backbone. The handhelds are loaded with software to protect from malware, viruses and worms. Harris designed them so they cannot gain access to census networks or personal information in the databases or other Census networks.

Harris hired an independent security firm to try to break into one of the handhelds using tools and skills an average user would possess. "They've followed all the best practices for security," says Jeff Waters, director of government solutions at Securify Inc. "It looks like it would be fairly difficult to break into this."

Still, the bureau's review board has about 15 items on its risk list, 60 percent fewer than Powner likes to see. The majority of the highest-level risks involve events that could increase project costs, cause deadlines to be missed or expand the performance requirements for the handhelds, according to Jack Marshall, Census' deputy project manager for FDCA, who heads the risk management strategy. "The high risks we have are the traditional ones for large IT projects: those related to [contract] requirements, security, development and of course funding," Marshall says. "Those are the big ones."

This attention to risk is good, as far as it goes. The danger comes when considering enterprise risks, which Census, like most organizations, failed to do. "On a project this size, I'd like to see a list of at least 100 risks, not 20 or 25," Charette says.

Such risks involve how new systems will integrate with legacy systems and with daily operations, which can be adversely affected when new technology moves in. Confusion and lack of a strong plan also can affect the quality of products and services or, in the case of Census, accuracy and completeness of data. Already some risks are becoming reality at the bureau.

'Bugs Are Showing Up'

Upstairs in a sparse meeting room at the Manna Church in Fayetteville, N.C., Kelli Hermesch, a 28-year-old mother of two, stands in front of 11 students, holding her Harris handheld.

It's May 9, the second week of the Census Bureau's address canvassing dress rehearsal. Eventually, about 700 newly trained enumerators will spread out across Fayetteville and surrounding counties using the handhelds to check the accuracy of the addresses in the mapping database and to add new addresses that didn't exist during the 2000 census. The same drill is being played out in Stockton, Calif., where 600 enumerators are going door to door. The bureau picked Fayetteville and Stockton to test the handhelds because they provide a cross-section of America, including large military populations and many non- English-speaking residents.

Hermesch, a crew leader who will oversee the 11 enumerators she is training, has been reading from a manual that contractor Harris wrote. She reads instructions for entering a new address. She stops and paces the floor while she waits for the students to log on to their handhelds. At least four are having trouble. "If they seem slow, don't keep tapping them," Hermesch advises. "It will freeze them up. We still have a few kinks; that's why we have practice."

For more than 10 minutes, Angela Cregg, Hermesch's assistant, leans over the table where Bernice Wolff, a 70-year-old retiree from Fayetteville, can't log in and is having trouble working through the menu for adding addresses. After a couple more minutes, Hermesch tells those who cannot log in to work with a partner.

During a break, Wolff, who like the majority of temporary enumerators the bureau will hire, is older and retired and therefore less technologically proficient than younger workers, says her handheld "is giving me a lot of problems. I put my finger on it [the biometric reader] and it says it hasn't found my finger, and then it locks me out for 15 minutes," she laments.

Wolff worked as an enumerator in 2000, hauling around hundreds of paper forms and large maps. "I like the old way because I'm old," she says. "I hope it will work, but maybe the paperwork that comes along with it will help me when I get stuck."

She isn't the only one having difficulty. Monique Moya, a crew leader for the Fort Bragg military base in Fayetteville, says that at any one time, at least two of the eight enumerators she supervises experienced some problem. Those in the field are instructed to contact crew leaders with problems. If crew bosses can't handle them, they are to call a Harris technician. Most of Moya's problems are software related or are caused by the 10 second or so communications delay with the satellite. "When the handhelds are working, it's great," she says. "But with this being the first rollout, all the bugs are showing up."

Hermesch says one of the crew leaders she trained-an older woman-quit because the technology was too intimidating. "People who are not used to computers have a harder time grasping the technology," Hermesch says. "It makes it difficult for them."

The Commerce Department's in-spector general and GAO warned that the handhelds risked causing confusion if Census failed to hire workers with computer skills. Nowhere on the list are risks associated with productivity or training. "So far, neither we nor the field people have applied such an impact," Census risk manager Marshall says. Census executives told GAO that if they used computer skills as a criterion for hiring, they would be unable to find enough enumerators.

If that's the case, say risk management experts, then Census should have developed a strategy to deal with the consequence of introducing new technology into a decades-old business process. That's enterprise risk.

"Any time new technology is introduced, productivity [tends to] decline . . . especially if the workers don't have computer skills," Charette says. "They should have planned for what was obviously going to happen."

Planning for Failure

Some risks are obvious, some are less so. Enterprise risks tend to be obscure because they require executives and IT managers to think across entire organizations. But one rises above all others and never should be left out of any planning: What if the new computer system doesn't work? What is your Plan B and, just as important, what's the plan for carrying it out?

Planning for a new system to fail is Project Management 101. Top executives should have a strategy to recover and continue operations if a new mission-critical system fails when it's brought online. For IT projects that replace legacy systems, project managers typically will keep the old system operational, ready to go back online if the new one fails.

The Census Bureau has no such plan.

In June 2006, Census Bureau Director C. Louis Kincannon appeared before the Senate Subcommittee on Federal Financial Management, Government Information, Federal Services and International Security to testify about the 2010 census. After sharp questions about rising costs, then-chairman Sen. Tom Coburn, R-Okla., asked Kincannon what the bureau planned to do if the handheld computers didn't work. The query, a Senate staffer recently recalled, was a "real softball question, one we thought Kincannon could answer easily and would look good doing it. But we could not believe his answer."

The director bluntly stated that the handhelds would work, implying there was no need for a contingency plan.

Coburn asked the question, in various forms, four more times, with Kincannon insisting the Census Bureau had proved that handhelds work and that there is no reason to plan for a contingency. (At the time, only three months had elapsed since Census awarded the handheld contract to Harris.)

Finally, an exasperated Coburn said, "Your testimony today is . . . there is no alternative plan if it does not work."

"We will have to hire more people to conduct traditional pencil-and-paper nonresponse follow-up," Kincannon replied.

The director "has turned risk-taking into a gamble. He's saying that there is a 100 percent chance the handhelds will work and nothing will go wrong. Nothing, especially in large IT projects, has a 100 percent chance of working. It's all a gamble now," says Charette.

Census' Marshall says part of the risk strategy for next year is to develop a continuity of operations plan, and he says Harris Corp. executives have it on their "radar screen. We've looked at contingency plans, but they aren't so formal," he says.

GAO's Powner says that in general, formulating contingency plans near the end of a project, especially a large IT project, is "too late. There's no going back and adding them in. It would be meaningless," he says.

It also is too late to plan for a possible failure when Census and Harris scale the system from its first field tests of 1,300 people to the 525,000 enumerators that will use the system in April 2010. Marshall says Census and Harris IT managers are thinking about the risk of scaling the system, but no tests or plans have been laid out.

Charette says the tests should have been conducted by now, including pushing the system 10 percent to 20 percent beyond its expected maximum workload, which could present complicated problems that need time to solve. "This is one of the biggest risks for a new system-increasing the workload from a small lab setting to a much larger real-life situation," Charette says.

"It's getting too late in the game to run these tests."

'Seat of Their Pants'

The Census Bureau's gamble might very well pay off. The handhelds could work flawlessly and the accuracy of data could be improved. After all, Census and Harris executives say they have experienced only minor software glitches and other problems they can fix quickly before the second dress rehearsal next year. And the bureau still has 32 months until the count kicks off in April 2010. But that's not the point. Changing the data collection process presented huge risks to the enterprise and the core mission of the bureau. Failing to manage the enterprise risks compounded them. Costs already are rising, according to GAO, with Census $31 million over budget on the handheld contract, according to the Senate's census oversight subcommittee as briefed by GAO.

What's more, Powner says, past experience with the decennial census has demonstrated that the largest cost increases occur in the last few months leading up to the count, which for 2010 would mean 2009. Moreover, requirements for the handhelds have increased five- to tenfold, according to the Senate oversight committee. "It's a huge red flag that contracts are in trouble when requirements in-crease that much," Charette says.

Sharon adds that "requirement creep" like that is a direct result of not properly analyzing what an organization is trying to accomplish and developing options to do that. "We are hugely concerned that they are flying by the seat of their pants and don't know want they want," says a Senate staffer.

More telling, perhaps, the savings Waite promised in 2001 are being eaten up by cost increases. Estimates now show that even with the handhelds, the bureau will end up with a more expensive census than originally projected, possibly costing $13 billion or more, Kincannon admitted to Coburn during last year's hearing.

By almost any measure, Sharon says, handhelds weren't a risk worth taking. The $445 million savings, spread across 10 years, represents only 1.5 percent a year, he points out. "That's absolutely not worth it, especially given the risks of the project," he says. If other options were considered back in 2001, money could have been better spent, he says.

Sharon says the project had little chance of generating savings in the first place because it was conceived and analyzed incorrectly: "The last place something like this should come from is somebody's head."