The Data Migration Challenge

ven as Congress began debating the creation of the Homeland Security Department, federal officials were contemplating how to gather the data and associated applications from the 22 agencies slated to move into the department in a cohesive, coherent way. Steve Cooper, special assistant to the president and chief information officer at the White House Office of Homeland Security, is leading the effort. He has the unenviable job of defining the proposed department's business objectives and needs, determining which existing applications and types of data can help the organization meet its goals, and migrating the data into a secure, usable, state-of-the-art framework.

Cooper views the job as an information integration challenge. "In this case, data migration is a big deal," he says. "In the human resources arena alone, we have identified 24 applications in the 22 agencies." It's possible the team will end up choosing three or four applications in vital areas-such as human resources, financial management and procurement-instead of just one. Still, "migrating the data and configuring three or four applications to address each area is not trivial," Cooper says.

Although the proposed Homeland Security Department is one of the most complex information-gathering and data migration projects under way in the federal government, the challenge of moving data from legacy systems, within or across agencies, is something all departments must address. Complicating the issue is the plethora of rapidly aging applications and databases throughout government.

As Cooper has discovered, the first step in migrating data is to identify all applications and data in use. Although it can be a mammoth job, even simple tools such as spreadsheets can help. More complex analytical and documentation tools available from such companies as Computer Associates, IBM, and BMC Software of Houston, also can be helpful.

The task of identifying applications can take time, dedication and perseverance. Nobody knows that better than Timothy Traverso, special assistant to the deputy Navy information officer. Traverso has the messy, time-consuming job of sifting through the 31,000 applications used by major Navy organizations and reducing them to a manageable number-ideally, less than 5,000-as part of the effort to develop the Navy-Marine Corps Intranet (NMCI).

After identifying applications and databases, the next step is to determine which to use and which to discard. In most cases, that decision is based on finding the ones that are the most functional, have the most complete information, and best adhere to modern technologies and standards.

Officials at the Office of Homeland Security plan to make such decisions by assigning each application and database a color-red, yellow or green-based on how challenging it would be to integrate them into the new framework. Green identifies reasonably modern applications that can be used fairly easily, while yellow connotes those that can be retrofitted and used with some work. Applications in the red zone, which generally are noncompliant or nonstandard, would take a significant investment of time and money to migrate to the desired platform. "Unfortunately, there isn't much green," Cooper says.

In some cases, the decision about which application or types of data to use isn't that difficult. "If we see that 80 percent of the agencies are on the same platform, it's a fairly easy decision," Cooper says.

Automated tools also can help agencies focus on the best platforms and databases. Two popular tools that can help with this task are the Genius Series from Glomark Corp. of Columbus, Ohio, and ROInow from CIOview of Boxborough, Mass. Both tools help organizations determine which alternatives provide the best return on investment.

MIGRATING THE DATA

Once an organization knows which data and applications it wants to keep, the difficult process of moving the data starts. First, organizations must identify a common thread in the data and build on it. That's where automated tools such as those from Candle Corp. of El Segundo, Calif., Computer Associates, and Tranxition Corp. of Beaverton, Ore.-not to mention homegrown tools based on the Extensible Markup Language (XML)-can help. These tools help make sense of disparate information, much of it referring to the same data in different ways.

"When you start drilling into data that is 10 or 20 years old, you might find some bad information, some of it corrupted," says Pete Scalone, area manager for federal systems in the Herndon, Va. office of Computer Associates. "So as you extract the data, you have to be sure you are extracting good data."

Conflicting information is one of the biggest dangers in data migration, says Renato DiPentima, president of Fairfax, Va.-based SRA International's consulting and systems integration business and former deputy commissioner for systems at the Social Security Administration.

"You would think there was one thing called a Social Security number-a nine-digit number with a basic set of codes behind it, with a basic set of edits behind that," he says. "But we found there were 16 different ways to define that number, because as systems grew over the years, some people called it a beneficiary number, some a claim number, some a folder number, some a payee number, and some a customer number."

Another major challenge in the data migration arena is security, especially when dealing with data and applications that are decades old. Traverso, for example, soon discovered that many of the Navy's thousands of applications had not been through the service's latest security accreditation and certification processes. The situation is compounded when data are pulled from multiple sources, especially those outside the confines of the host agency.

"A number of applications we have aren't classified, but the more information we gather can cause them to end up being classified from a network perspective," Traverso says.

The situation is much the same for the Office of Homeland Security, which also must deal with the added challenge of privacy, since the data migration project ultimately will include sensitive data from organizations such as the FBI and the Immigration and Naturalization Service. Cooper says his team is far from ready to deal with security issues, but he envisions using existing technology and protective measures whenever possible. Multiple levels of security must be set up, Cooper says.

"There is a difference between keeping the data secure and keeping the systems that house the data secure," says John Sleggs, director of defense and intelligence programs at NETSEC, a network security firm based in Woburn, Mass. "You have to look at whether moving data from existing to new systems in and of itself is secure and whether those new systems meet the security requirements mandated by whatever regulatory requirements that particular agency falls under."

Because of the importance and complexity of security today, especially with intra-agency data migration, many experts inside and outside government say it often makes sense to outsource the problem to a consultant, systems integrator or vendor that understands information security, systems security and federal policy.

DUE DILIGENCE

Although tools can help immensely, both in identifying and migrating data from legacy systems, they aren't the total solution. The time-consuming work of deciding on formats, policies, functionality and structure can only be done the old-fashioned way, by real people exchanging ideas.

"Before you get to any technology, you're talking about months of talks and coordination and format discussions," DiPentima says. "You have to ask yourself if you could wish it in place today, what would it look like and what would you need to have? What elements are you going to use, and from what system are you going to get them?"

That's the approach Cooper is taking in the homeland security effort. "We are spending a lot of time thinking it all through," Cooper says. In the case of the proposed Homeland Security Department, which aggregates information from 22 agencies, listening is as important as talking. To that end, Cooper and his staff have asked industry experts, especially companies that have been through massive mergers involving disparate systems and data, to share their knowledge.

"We're listening to private sector lessons learned that we can apply to the work we're doing," Cooper says. Officials from the Office of Homeland Security recently invited Carly Fiorino, CEO of Hewlett-Packard, to speak about this topic. "There were some valuable insights," Cooper says.

Traverso, who has been steeped in NMCI's application and data migration effort long enough to learn some valuable lessons, offers this advice: "Understand your IT environment before you attempt something like this," he says. "We never really understood our environment or what assets we really had. We thought the due diligence process required by the contractors for NMCI would identify that, but it really didn't."