In response to a question from Modern Healthcare about the change in course, a CMS spokesman said, “CMS appreciates the stakeholder feedback we’ve received on this particular statistical model and looks forward to additional feedback on the model.”
But hospital stakeholders and analysts said throwing out the model is an essential step to truly improve the star ratings.
“Moving away from the latent variable model and having a transparent weighing of the variables is foundational; it’s a key step,” said Dr. David Levine, senior vice president of advanced analytics and informatics at consultancy Vizient. “Of the suggested changes in the lengthy document, it’s one of the musts and it makes those other areas much more meaningful.”
The latent variable model is a statistical approach that gives more emphasis to certain measures over others based on a number of aspects, including variation in performance among hospitals for that measure or how much measures correlate to each other.
The CMS doesn’t decide how much weight is given to each measure; instead the model does it on its own. Because of this, the star ratings have been unpredictable, said Nancy Foster, vice president of quality and patient safety at the American Hospital Association.
In the July 2018 preview of the ratings, the latent variable model gave much more emphasis to hip and knee complication rates in the safety-of-care domain instead of the PSI-90 measure, which received the most emphasis in that domain in previous iterations of the ratings.
Unless the CMS removes such unpredictability, any other changes made to the program are unlikely to have a significant impact, Foster said.
“We think we know what might help, but unless you plug it through the latent variable model you don’t know if it’s going to do what you think it will,” she said.
The CMS also wants feedback on whether it should separate hospitals into peer groups, group measures differently, and release the ratings once a year. Comments are due by March 29.
Instead of the latent variable model, the CMS suggested assigning weights to each measure in the domains. There are currently 57 measures separated into seven domains that make up the total rating.
The agency said it would ask for stakeholder feedback to determine how the measures should be weighted, acknowledging that getting consensus “might be difficult to achieve.”
Maryellen Guinan, senior policy analyst at America’s Essential Hospitals, said she’s concerned that the CMS’ weights could prioritize some measures over others that stakeholders don’t agree with.
“You can see there are issues in terms of potentially unintended consequences,” she added.
While there is potential for disagreements, the process will be much more transparent than how it’s now, Hota suggested. “Right now, you have a statistical black box telling you the weights,” he said. “Having actual patients and experts telling you what the weight should be, that’s a much more democratic process.”
Hota and his Rush colleagues have been vocal about their issues with the star ratings, recently publishing a blog post that laid out their concerns.
In addition to potential changes to the latent variable model, Hota and others were also excited about the possibility that the star ratings could eventually be customized by users depending on their specific health needs. “I feel like we are at an inflection point where we are moving beyond these blunt measurement instruments and getting more precise,” Hota said.
The CMS said it would like feedback from stakeholders on creating a tool that would allow consumers to set their own weights for each of the seven measure groups used in the star ratings.
Dr. Mark Friedberg, a senior physician policy researcher at RAND Corp., said a customized tool for consumers is likely the future of healthcare rating sites but the industry isn’t there now.
“We don’t know how to do this yet,” he said. “It’s probably years away to get to a prime-time-ready version of this. It would be great if they (the CMS) take it up as a long-term goal.”
Friedberg, who created a mock webpage that allows the user to change the weights in the CMS star ratings, said it will require educating consumers about what they should care about for their condition.
Hota added that a customized tool would ideally have more precise measures, and he doesn’t think that’s far off. He pointed to efforts by the CMS to open patient data including the proposed rule last month that would require health plans to provide enrollees with access to medical information. The rule would also make public the names of providers that block patient information from insurers.
Open data-sharing may lead to much more precise, disease-specific measures that the star ratings now lack, Hota said.
“I feel like a real long-term plan is more personalized measures that help me with the things I’m struggling with, and not a one-size-fits-all approach,” he added.
