NSA spy program hinges on state-of-the-art technology

The furor over the National Security Agency's domestic eavesdropping, authorized by President Bush, has focused largely on legal questions -- whether the NSA has the authority to spy on Americans inside the United States and whether the commander-in-chief can order the agency to do so.

But that debate has largely smothered examination of how the nation's largest intelligence agency is collecting -- and analyzing -- information intercepted from hundreds, possibly thousands, of Americans. Since the 9/11 attacks, the NSA has abandoned the mantra that guided it in earlier decades -- Do not spy on Americans inside the nation's borders. Things have changed, and the NSA may be on the cusp of employing state-of-the-art technologies to uncover more information about potential terrorists, and about Americans here at home.

In the first days after 9/11, amid the palpable fear of another strike and an all-hands investigation of the attacks by the FBI and CIA, the NSA's then-director, Lt. Gen. Michael Hayden, took a broad view of his agency's power to conduct electronic eavesdropping.

Whereas existing laws, regulations, and executive orders restricted domestic monitoring of U.S. persons without a warrant, Hayden told House Intelligence Committee members on October 1, 2001, that he "had been operating since the September 11 attacks with an expansive view of [his] authorities," according to a declassified letter that Rep. Nancy Pelosi, D-Calif., then the committee's ranking Democrat, sent to the general after he briefed lawmakers.

Pelosi was troubled by the legal rationale for the NSA's activities. Although significant portions of her letter -- and most of Hayden's response -- are redacted, Pelosi wrote that she was "concerned whether, and to what extent, the [NSA] has received specific presidential authorization for the operations you are conducting." According to the letter, those operations included the NSA's sharing of intercepted communications with the FBI without first receiving a request for such reports -- the normal procedure, to avoid the co-mingling of intelligence and law enforcement operations.

According to sources who are knowledgeable about the NSA's domestic operations but who would not be identified because those operations are still classified, just after 9/11, the NSA targeted and intercepted the communications of specific foreign persons and groups, an indication that at least some of the targets were previously known to U.S. intelligence. The sources didn't specify whether any persons inside the United States were also monitored. But, Pelosi wrote, Hayden informed lawmakers that the NSA was making the call about what intercepted information was of "foreign-intelligence interest" before passing it to the FBI.

The New York Times reported this week that "in the anxious months after the September 11 attacks, the [NSA] began sending a steady stream of telephone numbers, e-mail addresses, and names to the FBI, in search of terrorists." Some of that information led to Americans inside the United States. It appears that the NSA was handing over just about any information it could find that might be useful to investigators. The Times reported that the NSA eventually provided thousands of tips a month.

The agency conducted these activities without presidential authorization for at least three months following 9/11. In early 2002, Bush authorized the current program, which, he has said, targets only known members of Al Qaeda and affiliated groups, and people linked to them. But even before Bush's order -- which remains classified -- the NSA's work was evolving from targeted interceptions to broader sifting and sorting of huge volumes of communications data.

Officials with some of the nation's leading telecommunications companies have said they gave the NSA access to their switches, the hubs through which enormous volumes of phone and e-mail traffic pass every day, to aid the agency's effort to determine exactly whom suspected Qaeda figures were calling in the United States and abroad and who else was calling those numbers. The NSA used the intercepts to construct webs of potentially interrelated persons. (The Times, citing FBI sources, reported that most of these tips led to dead ends or to innocent Americans.)

Analyzing large amounts of telecom traffic would give security officials valuable information about potential adversaries, revealing the times of day that terrorist suspects tended to conduct their communications, and the means they used -- land-line phones, mobile phones, or the Internet -- according to telecommunications experts.

One telecom executive told National Journal that NSA officials approached him shortly after the 9/11 attacks and insisted, to the point of questioning his company's patriotism, that executives hand over the company's "call detail records." Those documents, known as CDRs, trace the history of every call placed on a network, including a call's origin and destination, the time it started and ended, how long it lasted, and how it was routed through the network.

Having wholesale access to many companies' records would, in theory, give the NSA a picture of telecom usage across the country. And, since many U.S.-based carriers route international calls through their domestic switches, a picture of the wider world could emerge as well. The telecom executive said he believed that the NSA wanted the information to conduct a "data-mining" analysis of call and e-mail traffic.

Fifty years ago, intelligence officers had to manually scan communications intercepts for keywords, names, or other tantalizing intelligence nuggets. The NSA has long since automated that process with sophisticated supercomputers that can read huge stores of intercepts at speeds human beings can barely contemplate.

But more recently, the NSA has pursued cutting-edge data-mining technologies that don't just find key words but also uncover hidden relationships among data points. These technologies can even detect how a particular analyst thinks, identify what his or her biases are, and then suggest alternative hypotheses.

Data-mining systems, which the NSA has publicly pursued and spent millions of dollars researching, don't just "connect the dots" but also alert analysts about which dots to connect, which to disregard, and how to connect them in ways they may never have considered. It is unclear which, if any, of these data-mining tools the NSA is using to analyze the domestic information gathered in the current eavesdropping program, but the tools themselves offer a telling look into the agency's potential to exploit what it collects, regardless of its legal basis for doing so.

In September 2002, one year after the 9/11 attacks, a technology research-and-development office located at the NSA's Fort Meade, Md., headquarters awarded $64 million in research contracts for a new program called Novel Intelligence from Massive Data. The NIMD project, set to last for three and a half years, is intended to keep intelligence analysts from drowning under the massive flow of information they encounter and therefore missing key pieces of intelligence. In essence, it is an early-warning system.

"NIMD funds research to ... help analysts deal with information-overload, detect early indicators of strategic surprise, and avoid analytic errors," reads a "Call for 2005 Challenge Workshop Proposals" released by the Advanced Research and Development Activity (ARDA), the group at Fort Meade that was founded in 1998 to field new technologies for intelligence agencies, especially the NSA.

The administration has informed some lawmakers that the eavesdropping program the president authorized in 2002 is also designed to be an early-warning system. In late December, Assistant Attorney General William E. Moschella wrote to the top Democrats and Republicans on the House and Senate Intelligence committees, "The president determined that it was necessary following September 11 to create an early-warning detection system" to prevent more attacks.

Tellingly, Moschella wrote that the Foreign Intelligence Surveillance Act, which allows the government to obtain warrants to conduct domestic eavesdropping or wiretapping, "could not have provided the speed and agility required for the early-warning detection system."

The administration hasn't elaborated on why the system needs to operate independently of FISA, but officials may believe that it cannot meet the law's minimum threshold for surveillance, which requires a probable cause that the target is a terrorist, said Steven Aftergood, an expert on intelligence and government secrecy with the Federation of American Scientists.

"Logistically speaking, the early-warning approach may involve a significant increase in the number of surveillance actions," Aftergood said. "It may be that neither the Justice Department nor the [Foreign Intelligence Surveillance Court, which approves wiretapping warrants] is prepared to prepare and process several thousand additional FISA applications per year, beyond the 1,700 or so approved in 2004."

If the NSA is monitoring large numbers of communications -- The Times has reported that the agency has monitored as many as 500 Americans and other residents of the United States at one time -- then it stands to reason that applications for warrants could take time to process.

The NIMD project, as well as some others that ARDA is pursuing, closely resembles those of another controversial data-mining program aimed at discovering terrorist plots -- the Defense Department's Total Information Awareness program. Suspended in 2003, TIA was also designed as an early-warning system that would mine intelligence databases, but also private databases of credit card records and other transactions, for telltale signals of terrorist plots.

Of the companies and research institutions that won NIMD contracts in September 2002, six also held contracts for the earlier TIA project. Their TIA work focused on key areas of interest to NIMD, including challenging analytic assumptions and building prototype data-mining devices.

Like NIMD, TIA aimed to challenge analysts' traditional notions about what a given piece of intelligence might signify. It did this by creating a database of what TIA creator John Poindexter called "plausible futures," or likely terrorism scenarios. Another ARDA project also envisions such a database.

The Advanced Capabilities for Intelligence Analysis program, which is a cousin of NIMD, looks for ways "to construct and use plausible futures in order to provide additional, novel interpretations for today's collection" of intelligence information, according to the 2005 call for proposals.

TIA was distinct from NIMD and other projects in that it specifically focused on counter-terrorism, according to Tom Armour, a former program manager in Poindexter's office at the Defense Advanced Research Projects Agency. However, Armour says, the two research teams had "good coordination" and discussed their projects on a regular basis.

When Congress eliminated funding for most of Poindexter's projects, a number of them (the exact number is classified) were transferred to intelligence agencies. Armour and others associated with TIA would not disclose the names of those agencies, but a former Army intelligence analyst also involved in data mining and counter-terrorism confirms that TIA tools were transferred to other agencies, where work on them continues to this day.

Asked whether data-mining programs, such as NIMD, that the NSA may still be pursuing would be useful for analyzing large amounts of phone and e-mail traffic, Armour said, "Absolutely. That's, in fact, what the interest is." The former No. 2 official in Poindexter's office, Robert Popp, said that he and his colleagues wanted to know whether intercepted phone calls and e-mail would help find terrorists but not ensnare innocent people. "We didn't know," Popp said. "That was the hypothesis. That was the question that Poindexter and I wanted to do research on, to be better able to understand."

The similarities between TIA and the NSA's current data-mining operations were enough to prompt one senior lawmaker to signal his discomfort in a letter to Vice President Cheney. Sen. Jay Rockefeller IV, D-W.Va., the vice chairman of the Senate Intelligence Committee, was briefed by Cheney, Hayden, and then-Director of Central Intelligence George Tenet in July 2003.

"As I reflected on the meeting today," Rockefeller wrote, "John Poindexter's TIA project sprung to mind, exacerbating my concern regarding the direction the administration is moving with regard to security, technology, and surveillance."

Whether the NSA research shares another similarity with Poindexter's work remains, troublingly, unanswered. Poindexter's office spent between $4 million and $5 million researching technology and policy that would protect the privacy of innocent people whose names might turn up in a data search, Popp said. "No one else was, or is, to our knowledge, putting that kind of investment in the privacy R&D area, certainly not in 2002 and 2003, like we were."

The Senate Judiciary Committee plans to hold hearings in the coming weeks on the NSA's domestic operations. In addition to the specifics of how the NSA collects and mines information, senators undoubtedly will want to know what assurances American citizens have that they won't be ensnared in a vast data-search net.

Poindexter addressed the trade-off between privacy and security in 2003, when he was forced to resign as the TIA manager amid criticism that it was an Orwellian assault on civil liberties. In his resignation letter, Poindexter wrote, "It would be no good to solve the security problem and give up the privacy and civil liberties that make our country great."