The Leapfrog Group on Monday released its Fall 2016 Hospital Safety Grades, which evaluate the safety of thousands of general acute-care U.S. hospitals for patients. Every year, as many as 250,000 people are estimated to die as a result of potentially preventable medical errors.
“No hospital is perfectly safe,” said Erica Mobley, Leapfrog's director of communications. “Even the highest-scoring hospitals still have errors. If you look at the underlying data for the A hospitals, there are still some measures where they're struggling.”
The grades highlight successes and areas for improvement among individual hospitals. They also underscore the challenges and gaps in the very process of assessing hospitals for safety.
This fall, Leapfrog scored 2,633 hospitals, of which 844 earned As, 658 received Bs and 954 garnered Cs. Ds were assigned to 157 hospitals, and 20 received Fs.
Leapfrog, a Washington, D.C.-based not-for-profit founded by large employers and other healthcare purchasers, advocates for transparency in healthcare. Its hospital safety grades, released every April and October since 2012, are aimed at encouraging patients to pay attention to safety when they choose hospitals.
Ultimately, that will “foster strong market incentives for hospitals to make safety a priority,” the group said,
Some experts, like Michael Millenson, president of Health Quality Advisors, suggested that the impact of the report is limited.
“The general public has not responded,” Millenson said. “Purchasing managers have not responded, and hospitals sometimes respond.”
Leapfrog also ranks states by the proportion of A-rated hospitals they have. While states like North Carolina, now No. 5, and second-place Idaho have steadily climbed up the list, others, like New York, now No. 46, have fallen.
“Without knowing what's going on in each individual hospital, I think that when we see a big decline ... it's an indication of a hospital not making safety an institutional priority,” Mobley said.
Grades are constructed from a blend of process measures (hand hygiene is an example) and patient outcomes (think MRSA infections) derived from government data; Leapfrog's own hospital survey; and secondary data sources.
For fall 2016 scores, Leapfrog used data from the CMS reported from July 1, 2013, to June 30, 2015. It also took data from its own 2016 survey of hospitals, which is based on 2015 reporting. If hospitals did not fill out Leapfrog's survey, Leapfrog used data from the CMS Hospital Compare website or the American Hospital Association.
The 30 different measures, split evenly between processes and outcomes, are weighted to create a composite score. Hospitals are then graded in comparison to each other, rather than against an absolute score.
Although the Leapfrog Group aims at some point to grade hospitals on an absolute scale, “We just don't feel like we're quite there yet in terms of how hospitals are performing,” Mobley said.
One criticism previously leveled against Leapfrog's hospital safety scores is that because some of the data come from Leapfrog's own survey, hospitals that don't submit the survey, which by the group's own admission takes anywhere from 40 to 80 hours to complete, are penalized.
Thomas Rosenthal, chief medical officer for UCLA Ronald Reagan Medical Center in Los Angeles, told Modern Healthcare in late 2012, when his hospital received an F, that Leapfrog's methodology was “flawed.” The hospital had filled out Leapfrog's survey for grades released in the spring, but not the fall.
“We did it in June and said this is ridiculous to have an employee spend a substantial amount of time filling it out,” Rosenthal said at the time.
Editor's note: UCLA officials were not available to comment before this article was published. The medical center said in a subsequent statement that it "supports Leapfrog's efforts to drive improvements in healthcare quality" and noted that Leapfrog gave it a "B" in its fall 2016 grades.
The Hospital Safety Grade incorporates eight safe practice measures from Leapfrog's survey that are not collected elsewhere. If hospitals don't complete Leapfrog's survey, those measures are not included in their final grade. Instead, the weight assigned to those eight metrics is redistributed to other measures, Mobley said. Hospitals might score very poorly on those other measures—or do very well.
“So it's not if you report or not that can make a difference in your score,” Mobley said. “It's how well you're doing that makes a difference.”
Of the 40 to 80 hours of staff time Leapfrog's survey requires, “We recognize that that's not an insignificant commitment,” Mobley said, defending the process nonetheless. “Compared to the Joint Commission or registry reporting, it is significantly less,” she said.
Gauging abstractions like healthcare quality or hospital safety is not a perfect science. But according to Leah Binder, Leapfrog's CEO, it's better to use measures that are good enough to be useful to the public, rather than none at all simply because they remain imperfect.
“I think it's time for us to start to err on the side of imperfection,” Binder said in an interview with Modern Healthcare in early October. “The measures that are good need to be improved on over time ... but they should be improved in the sunlight. We shouldn't have to wait around for perfection, because our lives are on the line.”
