Netting Smiles

The Navy Marine Corps Intranet project continues to spawn user dissatisfaction even as a recent survey purported to show that nearly 80 percent of the network's customers are satisfied. The poll itself raised ire because it was conducted by the contractor responsible for developing the network and NMCI officials refused to release the survey questions.

Texas-based Defense contractor EDS is developing the system to connect all Navy and Marine Corps uniformed and civilian personnel. NMCI, the largest intranet in the world, could eventually cost the Navy more than $8 billion. EDS is responsible for developing the program and simultaneously measuring customer satisfaction. The results of the customer satisfaction survey are tied to financial incentives for the contractor.

EDS stands to reap financial rewards when NMCI customer service levels reach certain benchmarks, beginning at 85 percent. As customer satisfaction levels rise, EDS has the potential to earn as much as $100 per NMCI "seat" for each financial quarter, according to program officials. The Navy already has more than 200,000 active seats in the program and plans to move thousands more into NMCI in the next year. "There's a huge incentive there," says Rear Adm. Charles Munns, the outgoing NMCI chief. "A huge potential." EDS conducts quarterly surveys for the NMCI office, which recently reported almost 80 percent user satisfaction.

Outside pollsters say the NMCI survey arrangement is not in line with industry standards and could present a conflict of interest for EDS. "Usually there is a separation between the implementation and the evaluation process," says Cliff Zukin, a polling expert at the Eagleton Institute of Politics at Rutgers University and vice president of the American Association for Public Opinion Research. "If you are genuinely concerned with satisfaction, you would want an outside independent auditor, specifically if it involves incentives."

Eric Nielsen, senior director for media strategies at The Gallup Organization, agrees that an objective third party could give the NMCI customer satisfaction poll more credibility. "When there are financial incentives tied to achievement . . . I would want to look doubly hard," Nielsen says. "The way [NMCI] is structured, I would think that it would be good to have somebody else review it."

Navy officials, including Munns, have expressed complete confidence in the survey results and in the integrity of EDS. The intranet program office is now conducting its first audit of the survey. EDS declined to comment on the NMCI contract, or the customer satisfaction survey.

In contrast to the high customer satisfaction levels reported by EDS, disgruntled NMCI users continue to complain that the network is plagued by poor, unreliable connections and a slow delivery schedule. Unhappy users have cited a range of problems, from lost documents to an inability to obtain a remote connection. But the majority of criticism has centered on the speed of NMCI. Some personnel have complained that their productivity has been cut and that NMCI is significantly slower than the legacy systems it replaced.

Navy officials have responded to those complaints by saying problems are to be expected during the shakedown cruise of any new system. They also maintain that most dissatisfaction occurs within units or facilities recently brought into the network and that the number of complaints they receive is falling. They expect customer satisfaction levels to rise.

NMCI users have questioned the validity of the 80 percent customer satisfaction level. Those who have taken the survey say some questions do not relate directly to the system's performance. One user said he was asked whether the NMCI help desk personnel are polite. He answered in the affirmative, but couldn't make his point that they were not able to solve his problem. Despite controversy about the survey results, NMCI officials said last month that they will not make public the survey questions. During an NMCI press conference in early September, Navy officials said releasing the questions would compromise the program's ability to conduct future surveys to measure user opinion on the same issues over time.

NMCI officials said they would "strongly oppose" publishing the questions. "It is a bad practice to publish surveys in advance when you want people's current opinions," said Edward Schmitz, the senior NMCI customer satisfaction official. "The current set of questions [is] much easier to answer than previous surveys and provides useful high level diagnostic information on service quality."

Polling experts criticized the decision and encouraged NMCI officials to use an outside auditor to validate the survey results. They say the poll would carry more weight if the questions were released. "It's impossible to assess the results of the survey without knowing the questions," Zukin says. There is leeway when opinion polling is done within an organization, Neilsen says, but he also suggests it would be better to release the questions. "It is standard practice for us, with public opinion polls, to release not only the results but the questions," he says. "This being government, what is private is public."