Two scientific studies published last week cast doubt on the validity of basing quality standards on a hospital's annual volume of cases, undercutting efforts by the Leapfrog Group to use volume measures as a decisionmaking aid for consumers.
The studies examined the worth of directing patients to hospitals doing a high volume of certain types of medical care under the assumption that those facilities are likely to produce the best outcomes. The findings questioned the reliability and merit of using procedural volume to predict the likelihood of good results in heart surgery and in care of tiny infants.
"It adds to the evidence that there may not be a good relationship between volume and outcomes," said Carmela Coyle, American Hospital Association senior vice president of policy. "We have from the start questioned the use of volume measures."
The Leapfrog Group, a coalition of 150 large employers and public agencies, has promoted an initiative to foster "evidence-based" referrals for five surgical conditions, including coronary artery bypass graft surgery, and two neonatal conditions, including low-birth-weight babies.
Leapfrog's executive director said the business coalition will update its own base of research next month with new data that show a higher correlation between volume and outcome than the two studies published Jan. 14 in the Journal of the American Medical Association.
"There are competing analyses that demonstrate the effect of different sizes (of procedural volume)," said Suzanne Delbanco, who heads the Washington-based coalition.
Delbanco said Leapfrog eventually aims to report actual verified outcomes of the high-risk procedures for which it's compiling information. The use of volume-based standards is one way to prod the healthcare industry into gathering and reporting results of individual hospitals as a preferable alternative, she said.
And risk-adjusted mortality outcomes recently replaced volume as the primary basis for hospitals' scores on heart surgery, said Leapfrog spokeswoman Claire Turner. Hospitals participating in the coalition's Web-based reporting process would disclose data many of them report confidentially to the Society of Thoracic Surgeons National Cardiac Database, the same source used in one of the studies published in the journal.
When released next month, the additional studies commissioned by Leapfrog will justify the appropriateness of sharing volume-based referral information with consumers and purchasers, Delbanco said. "We're still confident that having volume information is better than having no information."
The detailed examinations by researchers at Duke University and the Rand Corp. that were in the journal reached the opposite conclusion.
In the study by the Duke Clinical Research Institute, hospital procedural volume was "only modestly associated" with death rates for coronary artery bypass graft surgery, lead researcher Eric Peterson said.
The study found wide variability in risk-adjusted mortality among hospitals of similar volume, and it identified many low-volume hospitals with low death rates as well as some high-volume centers with higher-than-expected mortality.
Similarly, the Rand study determined that hospital neonatal intensive-care units treating the largest volume of infants under three pounds at birth did not always have the lowest rates of infant deaths.
Access to better data on specific diseases from national registries, combined with better techniques for analysis, are leading to different conclusions, Peterson said. The Duke and Rand studies both used clinical results contributed voluntarily from hundreds of hospitals, which can be adjusted for severity of patient illness at admission far more accurately than the data based on hospital claims upon which most previous studies relied, Peterson said. The neonatal data were compiled by the Vermont Oxford Network, a cooperative that includes 40% of the nation's neonatal ICUs.
Once adjusted for severity of illness, some of the influence of volume immediately went away, Peterson said. Duke researchers also found that the correlation between volume and outcome was stronger among patients of Medicare age than for those under age 65. Most studies rely on the availability of Medicare data, but that limits the analysis to care of older and more frail people.
Using the thoracic society's clinical database of 267,000 procedures in 2000 and 2001, the study found the volume correlation "nonexistent" among patients younger than 65, Peterson said.
Because elderly people are less likely than those under 65 to seek out information about where to go for surgery, the contrast in the impact of volume between those two groups added to the pointlessness of the volume measure for selecting an individual hospital, he said.
"In general, patients who are critically ill or old are least likely to go to the Web and rationally make decisions," Peterson said. Conversely, younger patients more apt to seek out criteria for choosing a hospital would not be better off going to a higher-volume hospital, he noted.
Overall, the Duke study found hospital procedural volume was "almost worthless" in deciding between any two hospitals, Peterson said. The modest correlation for high-volume facilities was only discernible because researchers had more than 400 hospitals to work with, he said.
Among the 7,110 deaths related to heart surgery during the two-year study period, discrimination of referrals based on volume would have averted 50 deaths, or less than 1%. In addition, using volume as the sole criterion for referrals would unfairly divert cases from nearly half of low-volume medical centers with outcomes equal to or better than overall death rates, Peterson said.
The Rand study also found fault with thresholds established by Leapfrog to assign favorable ratings for care of tiny babies based on minimum volume. "There are some very good, high-quality small providers, and these standards would unfairly penalize those providers," said Jeanette Rogowski, lead author and a senior economist at Rand.
For low-birth-weight infants, "volume is not a reliable indicator of a high-quality provider," Rogowski said. The study of 332 hospitals over a five-year period found wide differences in death rates, but "volume explains very little of the variation," she said.
Basing a referral system on volume standards would have saved an estimated 11 lives annually among the study group, while basing referrals on historical mortality rates available in the Burlington, Vt.-based databank of neonatal ICU data would have saved 115 lives-a 10-fold difference. "We need better quality indicators than volume," Rogowski said.