The Value of Threat Levels (Or Lack Thereof)

Reihan Salam, over at his newish blog at National Review, has a very interesting post about his waning respect for former Department of Homeland Security Secretary Tom Ridge, and the recent brouhaha over the terrorist alert threat levels. Buried at the end, he has this important passage on evaluating the value of the alert system, something that's been discussed obliquely in the storm that's raging over Ridge's allegations that the system was politically motivated. Reihan writes:

How does one responsibly "score" something like a "threat level"? This is fundamentally a matter of political guesswork. And it's also about something we call "CYA." Ridge seems to believe that the Bush administration wanted the threat level raise to enhance the president's re-election prospects. But after the Bin Laden tape, many people believed that the Al Qaeda threat was real and potent. While Ridge claims that there was "absolutely no support for that position within our department," other administration officials clearly disagree. Moreover, the decision-makers were presumably mindful of the consequences of not raising the threat level and then facing a major terrorist attack. This is the reason why the threat level concept has always been utter nonsense.

I think this illustrates the difficulty of measuring the efficacy of many programs: you have different views of the desirability of programs, you have evaluative bodies that have politically appointed leaders, etc. It's really difficult, outside of the Government Accountability Office to even find ratings systems that focus on programs' workability, given how difficult it is to separate workability from desirability.