Hospitals see long road ahead in improving CMS star ratings

The decision two weeks ago by the CMS to delay the star ratings is in line with the Trump administration's larger goal to improve transparency in the industry. But analysts and hospitals still question how much meaningful improvement will be made to the ratings program.

The CMS last week put off for the third time an update of its overall hospital star ratings after hospitals raised concerns about the methodology. The CMS already had published results after a delay to improve the process last year, but those changes were generally panned.

The decision to delay also fits with HHS Secretary Alex Azar's price transparency goals. The CMS' proposed hospital inpatient payment rule would require hospitals to publish online a list of their standard charges in a machine-readable format and to update the information at least once a year. CMS Administrator Seema Verma has also advocated for a more consumer-friendly marketplace where patients have tools to make better healthcare choices.

Hospital leaders and analysts say these are all signs that the administration might be serious about making substantial changes to the star ratings. "It's very problematic to push a transparency agenda if the information you're pushing is viewed as flawed. It provides ammo to the opponents of transparency to say, 'See, we told you it's complicated and that meaningful and reliable transparency is a bridge too far.' So, it would behoove the administration to try and get this right," said Francois de Brantes, senior vice president of commercial business development at healthcare consultancy Remedy Partners.

But there is still skepticism about how far the CMS changes will go because of challenges around quality measurement and conflicting interests among hospitals. Michael Gentry, chief operating officer at Sentara Healthcare, said the industry still has a ways to go in quality measurement, so it will take awhile to get to a star ratings methodology that has the appropriate, meaningful measures for patients.

"My guess is this will be an ongoing stair-stepping process, and there will be more in the future. I don't see this as a hard turn to the left. I see this as ongoing incremental improvement," Gentry said.

The star ratings rely on hospitals' performance on 57 quality measures, but getting agreement on which ones to use and how to do so is difficult. "These things are imperfect, so they are smart to be listening to feedback and learning along the way," said Andy Slavitt, general partner of Town Hall Ventures and former administrator of the CMS. "There are two big mistakes when someone points out something should be improved: to ignore it or to overreact," Slavitt said.

Dr. Catherine MacLean, chief value medical officer at the Hospital for Special Surgery in New York, said she doesn't think those are the right measures to be using. The measures apply to different conditions, which makes for an "apples to oranges comparison," she said. MacLean advocates for a rating system that compares hospitals' performance on specific conditions. But such a system would require the development of more measures.

"We are going to have to get more measures, good measures. The problem isn't that there are too many measures, it's that they are bad measures. It's time to step back and reassess how we are going to do this," she said.

The CMS relies on feedback from hospitals to decide which measures to include in the star ratings program. De Brantes argues that hospitals don't advocate for measures that actually depict differences in quality performance between hospitals. He said that's apparent by the small number of measures used in the star ratings and the fact that no patient-reported outcome measures are included.

"It's the politics around quality measurement. When people complain it isn't fair, it's as fair as it can be based on the input, and if the input is poor, look at yourself in the mirror. Ninety percent of the time it's the industry giving input," he said.

But hospitals' desire for a better rating system is strong after some were shocked to see their star rating had dropped drastically from December 2017 to July. The proposed methodology increased the number of one-star hospitals from 260 hospitals in December to 338 hospitals in July. At the same time, the number of five-star hospitals rose slightly from 337 in December to 375 in July.

The significant changes see in the star ratings were likely caused by a shift in the safety group. As Modern Healthcare reported, the patient-safety grouping, which accounts for 22% of a hospital's total rating, heavily weighted the complication rate from hip and knee replacements measure in the proposed July release. In previous iterations of the star ratings, the PSI-90 measure was most heavily weighted in the safety group. Researchers at Rush University in Chicago took the time to outline the statistical problems with the star-rating process as it stands.

"I think because so many hospitals of different types and sizes were impacted with major changes—two-star differences over a short period of time—I'm hopeful this helps CMS see something bigger needs to be done around the methodology," said David Levine, senior vice president of advanced analytics and informatics at consultancy Vizient.

For some hospital leaders, the drastic changes they saw diminished their trust in the model. "When you are tampering with the model, it shouldn't change the results a whole lot. It speaks to an underlying lack of stability," said Mark Fontana, senior director of data science at the Hospital for Special Surgery. "Little differences shouldn't shift the results dramatically, one would hope."

The Hospital for Special Surgery received a five-star rating in the December release. The hospital declined to disclose its proposed July star rating.

Other hospital leaders think the latest delay demonstrates the star-rating model doesn't work and it's time to move on. "I'm hopeful we will get to better measurement and more useful data, but I'm worried that the overall structure of these star ratings is not feasible to move on from, so we may need to rethink how we try to report hospital quality and move away from these star ratings," said Dr. Karl Bilimoria, vice president of quality at Northwestern Medicine in Chicago.

"It's not a valid measure of hospital quality in most cases," he added.

The CMS is currently eliciting feedback from stakeholders on the future of the star ratings, including a public comment period. No date has been set for when the new ratings will be released.