Missing the Mark

Education gave out billions for software; now it says the programs don't work.

For more than five years, the Bush administration has poured billions of dollars into the education technology market, encouraging school districts to use high-tech software to comply with the rigorous testing and accountability standards of the 2002 No Child Left Behind Act. Now, it appears officials could be coming down with a case of buyer's remorse.

In early April, the Education Department announced the results of a two-year study of 16 leading reading and math software programs. The report found that the technology it had been supporting, in part through massive federal grants, did not significantly improve students' performance on standardized tests. The findings come as a serious blow to the $2 billion per year classroom technology industry, which traces its recent prosperity to the largesse of federal funds awarded to low performing school districts as a result of No Child Left Behind.

The landmark education bill, which is up for reauthorization in December, sets aside $700 million per year explicitly for education technology. Anxious to raise test scores and avoid federal sanctions, school districts feverishly spent their newfound windfall on a wave of new classroom-based technology systems.

Within months, math and reading programs played on Sony PlayStations and CDs boasting of comprehensive bilingual education began appearing in classrooms. Multimillion-dollar technology contracts were awarded to top industry players such as PLATO Learning of Bloomington, Minn.; Pearson Digital Learning of Scottsdale, Ariz.; Scholastic of New York; and LeapFrog SchoolHouse of Emeryville, Calif., whose products were included in the federal study.

In recent years, however, the technology funds have begun to evaporate. Faced with leaner budgets and mounting questions about the software's effectiveness, Congress cut the Enhancing Education Through Technology program-the source of most of the federal funds-from $700 million to just under $500 million in fiscal year 2005 and to $272 million in fiscal years 2006 and 2007. Now, with the release of the much-touted federal study, industry analysts suspect that school districts could start weaning themselves off classroom technology altogether.

Don Knezek, chief executive officer of the nonprofit International Society for Technology in Education in Washington, says the report could lead to an inhibition of federal education technology spending that would create a "drag on the industry of instructional software for reading and math; on instructional software in general, and on the use of technology in schools in general." The organization, which advocates for more spending on classroom technology, says the report was poorly designed and ineffectively administered.

The Education Department has been tight-lipped about its $10 million report, saying only that it is analyzing what impact the study's results could have on technology grants, but that a final decision has not been made.

The department's assessment will have major repercussions, not only for the software industry but for millions of students. More than three-quarters of all states rely on the technology program as their primary source of education software funding; 17 of them have no other outlet for purchasing high-tech teaching tools. In late March, 26 senators and 50 members of the House backed a plan to restore the education technology budget to its pre-2005 levels. The Bush administration, however, insists it will look to zero out the remainder of the program's funds next year.

While congressional appropriators haggle about technology budgets, educators and industry representatives are waging an equally fierce battle over the value of the software itself.

The study evaluated more than 9,400 students from 132 schools and found that "test scores were not significantly higher in classrooms using the reading and mathematics software products than those in control classrooms. In each of the four groups of products-reading in first grade and in fourth grade, mathematics in sixth grade and high school algebra-the evaluation found no significant differences in student achievement between the classrooms that used the technology products and classrooms that did not."

The study did not evaluate the efficacy of the products individually-a follow-up report will do that-adding that the findings were "not designed to reflect the effectiveness of educational technology across its entire spectrum of uses."

Despite the carefully worded caveat, the education technology industry came out swinging, maligning everything from the parameters of the study to the credibility of its authors. Mark Schneiderman, director of education policy for the Software and Information Industry Association in Washington, put the blame on schools, arguing that teachers lacked sufficient product training and students were not afforded enough time to work with the programs.

Nonetheless, Schneiderman says he is not concerned with the study, saying that "districts will look at how effectively the technology is working for them and at their own successes. If they are working well, schools will continue to use them."

Education officials agree that local results, not federal reports, will dictate future technology procurements. For example, in Florida's Palm Beach County, Riverdeep's Destination Reading program has paid dividends through improved student achievement on state reading and math exams, and the system remains a part of the district's core literacy program. "Nothing will ever replace teachers," says Richard Contartesi, Palm Beach's director of educational technology. "But these are key support tools."

For others, the results are less conclusive. In 2002, Chicago Public Schools spent $840,000 on early literacy software from LeapFrog SchoolHouse but it did little to boost the district's lagging test scores. Sharnell Jackson, the district's chief e-learning officer, doesn't blame the software, but rather how it was applied. "Just like a textbook or any other resource, this technology is a tool," she says. "And these tools can be misused."

That's a lesson administrators in Los Angeles learned the hard way. In 2001, the nation's second-largest school system, behind only New York City, spent $50 million for Pearson Digital's Waterford Early Reading System. Pearson recommends using Waterford for at least 30 minutes every day but Los Angeles, crunched for time, managed only half that. When test scores showed no significant improvement, the district pulled the software from its regular classrooms, limiting its use to after-school programs and scarce personalized training.

"I don't know if the technology doesn't work, but it needs to be used appropriately and monitored," says Ronni Ephraim, the district's chief instructional officer. "We are coming into an age of technology, and we need to find ways to use education technology well and appropriately."