The center had completed the Leapfrog survey for the period covered in the first round of grades, but hospital executives didn't think it was worth it to do it again for the survey used in this grading period. “We did it in June and said this is ridiculous to have an employee spend a substantial amount of time filling it out,” Rosenthal said. After receiving the F, Rosenthal said as an estimate he re-ran the medical center's grade results in the most recent period, filling in the missing Leapfrog data with the same numbers it used when it completed the survey during the summer. He said the medical center would jump to a C from its current F grade on that basis alone.
Leapfrog's survey results are a part of the Safety Score's process/structure group of measures, which make up half of a hospital's score, the other half being outcomes measures. The process category includes the use of eight measures that rely only on data collected in an optional survey by Leapfrog that together represent 22% of the total grade for hospitals that complete the survey, according to Leapfrog's methodology report. If a hospital did not turn in a survey for the relevant time period, that 22% weighting is redistributed proportionately to the seven other process/structural measures.
“I think what we do in redistributing the weights is extremely fair,” said Missy Danforth, senior director of hospital ratings for Leapfrog.
Leapfrog data does show that the 945 hospitals that reported to Leapfrog did better on hospital scores, but Danforth says it's not clear why that is. While 30.2% of all graded hospitals got an A, 46.6% of those that reported to Leapfrog were given an A. Danforth declined to attribute that to a self-selection bias in which hospitals that think they are doing well on the Leapfrog measures will choose to participate. She said it's possible that because those hospitals are reporting on and tracking those measures, they are working harder to improve them. “Once you start measuring something, you find you can improve on that,” Danforth said.
Leah Binder, Leapfrog president and CEO, said its nine-member oversight panel will continue to consider revisions of the scoring method based on comments it receives, continued review of the evidence and any additional research the panel feels is necessary.
Leapfrog revised its approach for two of the measures in response to industry comment in this round of grading. The panel includes hospital quality experts Dr. Lucian Leape, adjunct professor of health policy in the department of Health Policy and Management at Harvard University, Cambridge, Mass., and Dr. Peter Pronovost, senior vice president for patient safety and quality and director of the Armstrong Institute for Patient Safety and Quality at Johns Hopkins Medicine, Baltimore.
Regardless of how the results might be affected by the use of Leapfrog survey data, UCLA Ronald Reagan Medical Center and another hospital receiving an F are going to consider filling out the survey to try to improve their results. “We're discussing that. It feels like extortion,” Rosenthal said. “It's taking a resource (for something) that otherwise has no value.”
Executives for another sizable hospital that received an F and did not fill out the Leapfrog survey, 439-bed Phoebe Putney Memorial Hospital, Albany, Ga., are doing the same. “We probably will” complete the survey, said Dr. Douglas Patten, senior vice president of medical affairs for the hospital. Phoebe Putney did not fill out the Leapfrog survey because of “competing priorities,” one of which was handling the merger of the hospital with a former competitor.
Binder said Leapfrog encourages hospitals to fill out its survey, but the Hospital Safety Score grading system is about giving additional information to patients about the quality of hospitals, not encouraging participation in the survey.
Another complaint from Rosenthal and Patten was that hospital grades can be disproportionately affected by the care of as little as one patient. Rosenthal said UCLA Ronald Reagan lost a letter grade because of one patient with an air embolism in 2010.
Phoebe Putney's score was affected by two patients, Patten said. “One or two patient outcomes could make you look like a rock star or a cellar dweller,” Patten said.
Not all hospitals are complaining about the survey. “Leapfrog is a good source,” said Karen Briscoe, director of risk management for 457-bed Provena St. Joseph Medical Center, Joliet, Ill., which received an A Hospital Safety Score. It's one of the ratings that consumers can be looking at, she said. “I think there's no perfect system, but I don't think it could be ignored,” she said.