Looking the Wrong Way

When agency watchdogs miss the point, they stifle innovation, increase risk and perpetuate waste.

Agency inspectors general and auditors at the Government Accountability Office go to great lengths to promote efficiency in federal operations by detecting fraud, waste and mismanagement. Their findings are among the most power-ful catalysts for bringing about change for the good in government. But when they are wrong, that power to enable rapid action becomes in itself a source of waste and mismanagement. All too often, audit reports punish innovators because they are based on guidelines and checklists that fail to distinguish between the important and the trivial. As a result, these assessments can compel agencies to spend scarce resources on the wrong things.

This problem is especially common in addressing cybersecurity, an area of rapid change and complexity. Misguided audit reports can be the root cause of agencies' failure to implement important controls for computer network defense. Worse yet, they can prompt agencies to divert limited cybersecurity resources from real threats to less important work.

Such assessments miss the point of innovation. "It's like complaining about somebody who discovered a cure for cancer because it's not also a cure for the common cold." That is how Fred Schneider, a computer science professor at Cornell University and a member of the Information Security and Privacy Advisory Board for the National Institute of Standards and Technology, characterized a 2010 State Department IG report that concluded the agency's program for continuous monitoring of cyber threats was deficient.

The State Department initiative has received Senate and White House recognition as a model for other agencies, yet in July, GAO released an evaluation that echoes the 2010 inspector general report. GAO was deeply critical of the program, prompting government officials to question State's shift from triennial paper reporting on cybersecurity controls to continuous monitoring. GAO's report was seriously flawed and mischaracterized the security problem federal agencies face. Agencies and other auditors that rely on GAO's assessment of State's continuous monitoring program are sure to be misled about prioritization of controls for securing federal systems. The title of the report, "Information Security: State Has Taken Steps to Implement a Continuous Monitoring Application, but Key Challenges Remain," seems innocuous, but the practical effect is likely to weaken, not strengthen, the nation's cyber defenses.

Perhaps the report's most egregious oversight is that it failed to evaluate State's innovative system against the triennial reporting that most other agencies continue to rely on. Instead, GAO looked for gaps in the program's coverage and methodology, ignoring the enormous and unparalleled breakthrough it provided. Even if one accepts the accuracy of GAO's findings, its conclusions and recommendations to rein in continuous monitoring are inexplicable.

Strong evidence shows that the State Department has been far more effective at reducing risk and responding quickly to new threats than agencies that rely on the triennial process. And the department has spent less money on continuous monitoring than on the paper reports.

"One wasteful and ineffective area that [the Office of Management and Budget] and agencies can target is what is known as the certification and accreditation process-essentially a process whereby agencies evaluate every three years what defensive security protections are in place . . . The process costs tax-payers about $1.3 billion . . . on paperwork that ends up stored in binders in some clutter-filled room," Sen. Tom Carper, D-Del., said at a hearing in 2009. Carper, chairman of the Homeland Security and Governmental Affairs Subcommittee on Federal Financial Management, Government Information, Federal Services and International Security, requested the GAO study to determine whether the continuous monitoring system should replace triennial reports.

At a 2010 House hearing, then- federal Chief Information Officer Vivek Kundra admitted that the OMB-led "culture of compliance" needed to shift to a performance-based posture using continuous monitoring. "For too long, federal agencies have focused on reporting on security rather than gaining meaningful insight into their security postures," he said. "A well-designed and well-managed continuous monitoring program can effectively transform an otherwise static security control assessment and risk determination process into a dynamic process that provides essential, near real-time security status-related information."

The GAO report ignored the central question posed by Sen. Carper-whether continuous monitoring should immediately replace the triennial reporting system. GAO's failure to compare its effectiveness against what it is replacing is troublesome and misleading. Continuous monitoring is a key element of the Risk Management Framework published by NIST.

Since that framework was created, the complexity and persistence of attacks and attackers have forced continuous monitoring to the fore as the first and most important element of an effective risk management strategy.

Every working day, more than $1 million is wasted on triennial reports and other static security assessments. While it is not GAO's intent, its findings are being used as a delay tactic by people who like the status quo and others who exploit the system to rake in millions of dollars. If GAO adheres to its mission, then it will move quickly to correct its report and stop the waste and abuse it is fostering.

Franklin S. Reeder is a former Office of Management and Budget official and co-founder of the Center for Internet Security. He teaches and writes about information technology and policy.