Buying Tech Performance

For TSA, it's a tough climb to contract results.

In August 2002, the Transportation Security Administration contracted with Unisys Corp. to build a multibillion-dollar telecommunications and information technology infrastructure, a mammoth project to connect 429 airports and TSA's command center. The agency's mandated deadline to have the new security operations running at all airports? Three months later.

TSA opted for a performance-based contract, in which the agency provides overall goals in lieu of detailed specifications and pays the vendor to figure out the best way to accomplish them. What followed has been cited frequently by auditors and criticized on Capitol Hill and in the media for cost increases, low results and other problems.

A report released in February by the Homeland Security Department's inspector general (Transportation Security Administration's Information Technology Managed Services Contract, OIG-06-23) detailed problems with Unisys' IT Managed Services contract, known as ITMS. Among other things, it found the agency spent 83 percent of the initial $1 billion contract ceiling in less than half the allotted time without receiving many of the deliverables set out in the contract.

Those involved in the contract say that ceiling was meant to give potential bidders an idea of the project's scope-not as a maximum value-and that TSA originally estimated the contract would be worth several billion dollars. TSA also said DHS added $106 million in work orders, out of $940 million spent through fiscal 2005, for IT support and US VISIT, a program requiring visitors with nonimmigrant visas to give biographic, travel and biometric information before entering the country.

Despite problems unearthed by the IG, proponents of performance-based contracts say ITMS is not an indictment of the approach, but instead teaches important lessons about how to properly execute it. Performance-based procurement means that success is measured by overall real-world results, not by mere compliance with discrete contract specifications.

TSA and others say that because the agency was trying to stand itself up as it worked to meet tight congressional deadlines for airport security, it had no option but to contract for a set of objectives rather than narrowly specified outputs. "For an infant agency with no technology infrastructure and practically no IT or acquisition staff, a managed services contract using a statement of objectives approach offered the best hope," wrote Pat Schambach and Elaine Duke, TSA's chief information officer and chief procurement executive at the time, in an article in the winter 2003 issue of The Public Manager magazine. "For a chief information officer with few staff-and an acquisition office with one IT contracting officer and an acting director-it seemed the only way to proceed."

Indeed, proponents say managing performance-based contracts requires fewer people because traditional contracts are much more detailed. The problem is that the method requires skills those with traditional procurement training often do not have. "Managing a performance-based contract is less people-intensive than managing a traditional type of contract. On the other hand, it is true the traditional procurement workforce is not well-trained to do this kind of procurement model," says Steven Kelman, Weatherhead Professor of Public Management at Harvard University's John F. Kennedy School of Government and a former administrator of the Office for Federal Procurement Policy. Still, Kelman says the required skills are not "so arcane that people couldn't learn them."

Procurement personnel must make sure the statement of objectives, although broad, accurately captures the government's needs-a sample objective from the ITMS contract called for "highly reliable, and secure, IT managed services and support that meets or exceeds customer requirements and expectations." They also must find a way to measure the contractor's success, which can be more difficult than it seems. "If the vendor says, 'I guarantee 90 percent customer satisfaction'-is that a lot or a little?" Kelman asks.

A lack of adequate measurement tools was one of the ITMS problems identified in the February report. The agency "did not establish or implement adequate performance measures at the beginning of the Unisys contract," the IG found. "Performance measures have evolved and improved over the life of the contract, but they were limited . . . and added too late in the contract process to have an impact."

The other lesson from ITMS is the importance of being ready to manage a performance-based contract before it's awarded, says Chip Mather, co-founder of Acquisition Solutions of Arlington, Va., which helped TSA on its acquisition strategy for ITMS. "They really have to start putting [management tools] in place when you start the acquisition, so when the contract is awarded, they're there and you're ready to start managing," he says. "You can't wait until the contract's in place to try to catch up with the train that just left the station."

The IG report says its recommendations have been resolved, noting that TSA re-competed some elements of ITMS and implemented practices to address weaknesses. Kelman and Mather say much of the criticism has been overblown, considering the deadline pressure TSA was under. "They wouldn't have had anything if they had tried to do it in the traditional [contracting] route," Mather says. "If you understand the challenges . . . I think most people would step back and say it was miraculous that they got as much [performance] as they got."

Performance-Based Prep

Procurement personnel and program managers should be ready to run performance-based contracts before they are awarded, says Chip Mather, co-founder of Acquisition Solutions of Arlington, Va.

Successful management depends on six elements, he says:

• A structure for who makes decisions when.
• A plan for communicating with everyone involved.
• A link to overall strategy to ensure that changes are consistent with goals.
• Cultural transformation, so the contractor and government work as a team.
• Techniques to identify and deal with risk.
• Tools and metrics to measure the contractor's performance.