CMS changes to hospital star ratings don't address concerns

The changes to the star ratings methodology, which was released to hospitals on Friday in a preview report, were minimal as the CMS maintains its use of the latent variable model, which analysts and hospitals have taken issue with for the way it assigns hospitals star ratings.

The CMS delayed releasing the star ratings earlier this year when its July preview of the star ratings showed 98% of the safety-of-care domain relied on hospitals' complication rates from hip and knee replacements. It resulted in some hospitals seeing big changes in their ratings from December 2017. The new methodology flips the safety-of-care domain back to its original iterations in which PSI-90, or the Patient Safety and Adverse Events Composite, is most heavily weighted, at around 70%.

"We see these changes as relatively minor in the scheme of things," said Nancy Foster, vice president for quality and patient safety at the American Hospital Association. "We hope they are looking at making substantial further refinements in this methodology going forward."

The agency calculates star ratings by weighing how hospitals perform in seven categories: mortality, safety-of-care, readmission, patient experience, effectiveness of care, timeliness of care and efficient use of medical imaging. The three outcome groups—readmissions, safety and mortality—are weighted the most at 22% each along with the patient experience group. As Modern Healthcare reported, these heavily weighted groups can lower a hospital's overall rating.

Using the latent variable model, measures within each group are assigned loading coefficients to calculate how much that measure will influence how a hospital performs in each category. The latent variable model is a statistical approach that gives more emphasis to certain measures over others based on a number of aspects including variation in performance for that measure or how much measures correlate to each other. The CMS does not choose which measure to emphasize. Instead the model chooses which measure to emphasize.

The CMS supports the model because it's flexible enough to "accommodate changes in measures over time," a CMS senior official said in a July interview with Modern Healthcare.

It's unclear why the loading coefficients in the safety-of-care domain reverted back to previous iterations where PSI-90 is assigned the greatest loading because the CMS' most recent methodology changes don't appear to address the issue, said David Levine, group senior vice president of advanced analytics and product management at Vizient.

The CMS said it made two changes to the methodology. The first was the "removal of measures with statistically significant negative loadings" from the latent variable model. The other change was weighing healthcare-associated infection measures in the safety-of-care group differently.

But, as Levine points out, the PSI-90 measure and the complication rates from hip and knee replacements measure never had negative loading coefficients. Additionally, the healthcare-associated infection measures are assigned such small loading coefficients by the model that they're unlikely to sway how hospitals perform in the safety-of-care domain.

"My biggest beef is this is still dependent on this latent variable model which inherently allows the computer every time to weigh things different," Levine said. "It's like I have something wrong with my electricity and I do a patch. Eventually it's going to malfunction again."

In the same July interview with Modern Healthcare, the CMS senior official said the methodology assigning the heaviest loading to the PSI-90 measure isn't necessarily "a good or bad thing." She said the PSI-90 measure most accurately reflects safety of care in a facility because it's a composite of multiple measures of rates on pressure ulcers and sepsis that affect many patients.

But Foster at AHA disagrees that the PSI-90 measure should be getting so much emphasis under the safety-of-care domain. "The PSI-90 measure is one of the least reliable measures I have ever seen in my career working on quality measures and for it to be so heavily weighted in this domain and to have such a strong influence on the star ratings is a real mistake."

Akin Demehin, director of quality policy at the AHA, added that PSI-90 relies on claims data so it's less actionable for providers to improve upon. Furthermore, the data on their performance for the measure are about 2 to 3 years old.

Another problem is the readmissions category doesn't use the same risk stratification for the readmission measures the CMS now uses for its Hospital Readmissions Reduction Program. Under the program, hospitals are now divided into five peer groups to address differences in socio-economic conditions between hospital patient populations.

"At a minimum, we would like the CMS to consider implementing some of the socio-economic adjustment for the readmission measures since that is already in play" in the readmissions reduction program, Demehin said.

For the February release, the CMS expects 3,725 hospitals will receive a star rating, meaning out of the country's more than 5,500 hospitals, only 3,725 offer services measured by the star ratings. Similar to previous years, the distribution of hospitals with ratings of one to five stars resemble a bell curve. There will be 282 hospitals that will receive one star and 293 that receive five stars. The majority of the hospitals fall in the middle, receiving two, three or four stars.

Hospitals and other stakeholders have until the public release in February to offer feedback on the CMS' changes.