In the NEJM article, the RAND researchers argued hospital ratings sites don't have to create unchangeable methods. Technology now allows the sites to reform their methodologies so the weight given to quality measures used to determine overall scores can be changed depending on the patients' needs.
"Now that we have the internet, people can fiddle with the knobs — they can get something that is customizable to them," said Dr. Mark Friedberg, an author of the paper and senior physician policy researcher at RAND. "There is no reason not do that because they (rating sites) don't have to take the methodologically fraught step of trying to decide how much truly different things should matter."
RAND created a mock web page using 2016 data from the CMS' overall hospital star ratings to demonstrate how it would work. The web page allows patients to change the weight given to the seven quality domains the CMS uses to determine a hospital rating: mortality; safety of care; readmissions; patient experience; timeliness of care; effectiveness of care and efficient use of medical imaging. It also allows the user to see how their changes compare to the hospital star rating assigned using the CMS methodology.
Friedberg said the CMS hospital star ratings were only used because the methodology and data is publicly available so it can be easily replicated, but the same experiment can be applied to other rating sites.
There are many scenarios in which a patient might value some quality measures over others, according to RAND's analysis. For instance, a 45-year-old patient in West Covina, Calif., wondering where he should get his elective knee surgery performed may determine that he values safety and readmissions more than effective use of imaging or timeliness. His modifications change the star rating of Chino Valley Medical Center from four stars to five, while it demotes Methodist Hospital of Southern California from five stars to four.
"I don't want to pretend the way the tool is right now is where it should be for prime-time use … we are just trying to illustrate how highly sensitive to the weights the ratings are, and it's not implausible people might have different ratings," Friedberg said.
But some experts are concerned over how much control consumers should have in determining which quality measures are important for certain procedures.
For someone to use this tool "they would have to know how to interpret the measures meaningfully ... most consumers I don't think will have that level of expertise," said Chris Duke, director of consumer engagement programs at Altarum.
To address that, Friedberg said the ratings sites can offer suggestions or templates to consumers of how measures should be weighted depending on the procedure they are shopping for. Duke agreed that could address the concern if it's done in a consumer-friendly format.
Another concern is the fact that the tool relies on flawed quality measures already used by Hospital Compare, said David Levine, senior vice president of advanced analytics and informatics at consultancy Vizient.
The CMS star ratings don't necessarily use measures that patients care about. For example, there are no patient-reported outcome measures like functionality after a procedure.
"This is an innovative approach that is built on bad and flawed data that could be as harmful or more harmful to a patient making a decision," Levine said.
There are ongoing efforts in the industry to improve quality measures that are more meaningful to patients. Once that work is solidified, this tool can be a great benefit to patients, Levine said. "That is where this approach would work well," he said.
