VA medical system earns high customer satisfaction ratings

Good news about government is rare enough. What's outright shocking is when that good news is about how nice government employees are. But that is the bottom line of the recently released American Customer Satisfaction Index report on the Department of Veterans Affairs' medical system.

Not only do veterans rate their VA care much higher overall than the general population rates its hospital experiences, vets also consistently give VA doctors and nurses high scores for "responsiveness" (83 out of a possible 100), courtesy (90 out of a 100), and "respect and dignity afforded patients" (91).

For a massive federal bureaucracy long perceived as the health care equivalent of the Department of Motor Vehicles, the independent ACSI results are a great vindication.

"The American people have no idea what an outstanding organization they're funding for veterans," VA Secretary Jim Nicholson told National Journal, in an interview requested by the department as it touted the report to the media. The ACSI scores show, Nicholson said, that "the caregivers in the VA health system are very competent and compassionate, and the president and Congress are giving them the resources to do their job."

But not so fast, say veterans' advocates. VA care may be great, but veterans' access to that care is not. "Once you get into the system, they treat very you well," said Richard Fuller, legislative director of the Paralyzed Veterans of America. The problem is getting in.

During 2005, some 263,257 veterans who applied for VA care were refused enrollment because the department classified them as "Priority Group 8": That is, they had no disabilities caused by their time in uniform and had annual incomes above a (geographically adjusted) means test of about $25,000. That cutoff was imposed in January 2003 to preserve scarce resources for disabled and impoverished vets (Priority Groups 1 through 7).

And of the applicants who were let in, the VA's latest data show that some 22,000 newly enrolled veterans were still waiting to get their first appointment scheduled with a doctor. Thus the waiting list remains a chronic complaint, although the number is well down from a peak of 176,000 wait-listed in 2002, before the VA stopped enrolling new Group 8 vets.

The ACSI researchers, by design, did not talk to anyone still on that wait-list, let alone to anyone refused enrollment altogether. They randomly selected veterans who had just completed a course of care and asked them more than a dozen questions about the care itself, but none about any frustrations beforehand.

"We're not capturing that," said David van Amburg, who directs the ACSI program for the University of Michigan. "We're looking at the very end of the process."

That focus makes sense in the context in which both commercial clients and, since 1999, government agencies have always commissioned ACSI research: self-diagnosis. The VA already had plenty of data on its wait-list problem, procedures performed, patient outcomes, and more, but the hardest thing to quantify was customer satisfaction, which is ACSI's strong suit.

So the ACSI scores tell only part of the story of today's VA. The flip side of all those satisfied customers is more and more veterans wanting to use the system, including thousands who have other, less generous private coverage.

The Bush administration, backed by Congress, has balanced demand and supply by hiking the VA's health budget with one hand -- "up 57 percent under this president," Nicholson noted -- and turning away the less-needy "Priority 8" veterans with the other. But both the excellent care and the unmanageable demand were inherited from the Clinton administration.

It was Vice President Gore, busily "reinventing government," who in 1999 ordered 30 federal programs (ranging from Head Start and Medicare to the Park Service and the IRS) to employ the American Customer Satisfaction Index, already in widespread use in the private sector. President Bush, however, made the scoring optional, and although other agencies have since adopted ACSI on their own, only six of the original 30 programs have stuck consistently with the survey since 1999.

The VA health system embraced outside scrutiny to reinforce a revolution it had already begun. "We were doing [other] customer satisfaction surveys" as early as 1994, recalled Dr. Kenneth Kizer, VA undersecretary for health from 1995 to 1999, but ACSI was arguably "a better instrument.... I thought it was important that we could compare ourselves to the private sector: Is the American taxpayer getting a good return on investment? Let's see what customers think."

Under Kizer, the VA had closed many of its costly inpatient wards -- alarming members of Congress whose districts had lost hospitals -- and opened a network of cheaper outpatient clinics -- alarming budgeteers who warned, presciently, that increased convenience would increase demand and drown any savings. To manage itself for maximum efficiency, the VA gave every patient an electronic medical record to control costs, track test results, and catch medical errors (a system still unmatched in the private sector).

In this environment, an outside customer service survey was not a threat to the VA's leaders, but a weapon they could wield to force reform. Veterans' groups forced the controversial Kizer out of office before the first ACSI report came in.

The 1999 study, however, scored the VA's new outpatient clinics at 79 out of a possible 100 (inpatient care was not scored until 2001), well above most private hospitals. Since then, despite a statistically small drop in the scores from 2004 to 2005, the gap between the VA and private sector has only grown.

Just how comparable are these two medical systems, though? The overall "satisfaction" score is derived from three core questions: How satisfied were you? How did your experience compare with your expectations? How did it compare with your ideal of a perfect experience? The survey then asks these three questions in the exact same way across all the 200-plus companies in more than 40 industries, from carmakers to health care to soft drinks.

"We typically asked three or more questions for each item measured," to guard against statistical anomalies and allow elaborate double-checking, explained ACSI inventor Claes Fornell.

Fornell based the survey's sophisticated techniques on his work as a military intelligence analyst in his native Sweden, where he cross-referenced sonar readings, reconnaissance photos, and other fragments of data -- each piece ambiguous on its own -- to create a reliable picture of where Soviet submarines were lurking in the Baltic.

Fornell and his colleague van Amburg are confident that their sophisticated statistics give comparable scores for the private sector and the government, up to a point. Outside the three core questions, ACSI tailors its survey to each client's needs.

For example, the chronically poor ratings for emergency-room care depress the overall score for private-sector hospitals. Yet the survey doesn't score the VA's emergency care at all, nor does the VA offer much emergency care. ACSI makes sure it interviews a typical cross section of each client's customers, but it does not correct for differences in expectations between the VA patients -- averaging over 68 years old and often schooled by World War II and the Great Depression not to complain -- and the public using general health care.

Neither does ACSI look for seasonal spikes in health care demand from the flu, colds, and "snowbirding" by retired veterans who spend the winter in Southern states: Every survey of the VA has been done in the summer. ("The VA may want to correct for that," Fornell said.) And, of course, the survey does not ask about the hot-button issue of access at all.

No methodology can get answers to every question. What the ACSI survey is designed to measure, it measures well. The care provided really is that good, said Dr. Gail Wilensky, who chaired a presidential task force on veterans' health, "and the VA should take great pride."

But even the best statistics cannot answer the larger question of whether the country owes all veterans that standard of care, regardless of the cost, or whether it should ration a limited supply among the neediest veterans. Those are questions of values for Congress to decide.