The use of confidential, in-house feedback in the form of report cards linked to hospital administrative data is "unlikely to be a sufficient strategy for healthcare quality improvement," according to a report in the current issue of the Journal of the American Medical Association.
Researchers at McGill University Health Centre in Montreal measured how many elderly patients who survived an acute myocardial infarction filled a prescription for a beta-blocker within 30 days of being discharged from 76 hospitals in the province of Quebec between April 1, 1999 and March 31, 2000. Thirty-eight of the hospitals received "rapid feedback" in the form of report cards as soon as the data was randomized (May 2002), and 38 received report cards 14 months later (July 2003). For comparison, the same data was recorded at the same hospitals for similar patients between Oct. 1 2002 and March 31, 2003.
Compliance with this common quality measure rose at the rapid-feedback hospitals by 9.6%. The researchers, however, stated that this was statistically insignificant in light of the fact that compliance at the delayed feedback hospitals rose by 5.4%.
Louise Pilote, M.D., one of the study's authors, said in an e-mail that the study will spark interest in evaluating hospital report cards. "Given that it is a negative study, public reporting will have to be enhanced with other measures to improve quality," she said.
Todd Ketch, vice president of government affairs for the American Health Quality Association, said the study shows how "simply providing report cards -- especially on a single, one-time basis -- doesn't achieve the desired improvement they (hospitals) are pursuing."
"It's not always as simple as telling providers 'You have a problem here,' because they may not have all the information required to make the improvements they need," Ketch said. "We've seen better results when (the information) is coupled with support."
He also noted that even the rapid feedback group received data that was 2 years old and that "timeliness of data affects how it's used and viewed by providers."
Ketch added that the introduction of more healthcare information technology will make quality-improvement efforts move faster as physicians can view real-time data, make improvements and see the results.
Nancy Foster, vice president for quality and patient-safety policy at the American Hospital Association, said that while the study doesn't really make a statement on whether public or private reporting does more to improve quality, it's findings are still useful.
"It supports some earlier work that showed a one-time intervention of any sort is not an effective way to achieve sustained improvement in the quality of care being offered," Foster said. "I think that is the central theme of the study."
In that respect, Foster said that it doesn't matter that the study was conducted in Canada and not in the U.S. "I think the results are very transferable," she said.
The report noted that one reason Quebec was used as the setting for the study was because it is a "report card-naive region," and that most practitioners there had little experience with them.
In an editorial accompanying the report, Eric Peterson, M.D., associate professor of medicine at Duke University Medical Center in Durham, N.C., said that some physicians might question the use of administrative data instead of chart review.
The researchers acknowledged this in their report and noted that another study is under way, this one looking at hospitals in the province of Ontario and measuring whether it makes a difference if report cards are developed using chart review data. "This remains an unsettled question which will be answered by another study," Pilote said. "But I suspect the impact will not be much better than that with administrative data."
Pilote said she expects the study to be published next year.
Peterson also wrote that some physicians might object to using data on whether patients filled a prescription as a quality measure because patient compliance is beyond a hospital's control. "On the other hand, secondary prevention therapies are only effective if actually used," Peterson added.
According to Paul Keckley, executive director of the Vanderbilt Center for Evidence-based Medicine, Nashville, hospitals and physicians do have some control over patient compliance with prescription regimens.
"The data we have suggests that physicians have to alert patients within 48 hours of a diagnosis to remind them of the treatment directive," Keckley said. "And then, optimally, weekly for at least 30 days to create an expectation the doctor is watching and adherence is important."