GAO: Army needs to demonstrate accuracy of workforce projection model

The Army has not properly demonstrated that a mathematical model used to predict future civilian workforce needs produces on-target projections, according to a new report from the General Accounting Office.

Since 1990, the Army has used its Civilian Forecasting System (CIVFORS) to aid in strategic hiring recruitment and hiring decisions. The model uses data, including an employee's date of hire, age and educational background, to project retention rates and highlight areas where the Army might have gaps in skills in the future.

Army managers have done a good job of ensuring that personnel data entered into CIVFORS is accurate and that the information technology system housing the model works properly, the report (GAO-03-1046) said. For instance, Army employees edit historical data once it is entered into the model. The CIVFORS system will also alert employees to entries that look suspicious, such as those containing "inappropriate numbers or characters."

Since the model's launch in 1987, the Army has asked independent and internal reviewers to check its accuracy. But during a review conducted from September 2002 to June 2003, GAO researchers found that the Army had not completed a comprehensive assessment of the model's forecasting capabilities since its induction.

The model has changed several times since 1987, GAO said, necessitating a thorough assessment to reestablish its credibility. If the Army has conducted such an assessment, it has not documented its efforts adequately, GAO noted.

Other federal agencies have considered developing similar models, the report said, making it especially important for the Army to set a good example.

"There exists a risk that the forecasts [CIVFORS] produces may be inaccurate or misleading," GAO warned. "Furthermore, without documentation of CIVFORS' forecasting capability, it may be difficult for the Defense Department and other federal organizations to accurately determine its suitability for their use."

But in a letter responding to the report, the Defense Department said GAO has no reason to doubt the model's forecasting abilities. "The ultimate test of the system is performance," wrote Charles Abell, principal deputy at the Defense Department, in response to the report. "The CIVFORS system has been consistently generating Army projections with high standards of accuracy."

The personnel model has accurately projected trends in the Army's civilian workforce since fiscal 1990, and has exceeded standards set by other government and private sector models, Abell wrote. For instance, the Army's civilian model predicted a 222,600 workforce in fiscal 2002-a prediction that fell 849 people short of the actual workforce of 223,449.

This prediction deviated by less than four-tenths of a percentage point from the actual number, whereas typical private sector and government forecasting models produce predictions that vary from actual statistics by 2.5 to 5 percentage points, Defense noted.