CMS' updated star ratings formula still dings safety-net providers

The CMS recently updated the star ratings on Hospital Compare using a new methodology in an attempt to improve accuracy and fairness. Yet a dive into how hospitals were scored shows the same issue persists in which large, tertiary hospitals didn't perform as well as community facilities.

Using the new methodology looking at 2017 data, 337 hospitals have a five-star rating, and the majority are community hospitals. For example, Brigham and Women's Faulkner Hospital, a 162-bed community hospital, received a five-star rating, while its 777-bed flagship affiliate, Brigham and Women's Hospital, received a four-star rating. UCLA Medical Center in Los Angeles, which was named the seventh-best hospital in the country by U.S. News & World Report, has a three-star rating on Hospital Compare. Yet UCLA's medical center in Santa Monica did receive a five-star rating.

Industry stakeholders, especially hospitals, have criticized the star ratings for disproportionately assigning lower star ratings to safety-net providers versus non-safety-net providers.

"We still have some of the same trends and issues we had before," said David Nerenz, the director of the Center for Health Policy and Health Services Research at the Henry Ford Health System in Detroit. "Small specialty hospitals or hospitals in affluent suburban areas are overrepresented in the five-star list."

The CMS had postponed the release of the star ratings since July as it worked to improve the methodology in response to criticism that the previous formula provided inaccurate information about providers. Although industry experts say the new methodology is improved, the star ratings are still flawed.

Consultant Rita Numerof commended the CMS for its removal of winsorization, a tactic in which extreme outliers in the data set are trimmed to minimize their effect on the overall ratings. Instead, the CMS will run k-means clustering "to complete convergence." This involves clustering hospitals repeatedly into five groups—or star categories—until the hospitals in each group are more similar to each other and different enough from the other four groups.

Numerof said she is still concerned the star ratings normalize poor quality performance. The CMS compares the performance of hospitals to each other to get the composite star rating. For hospitals that don't report all measures involved to get a composite star rating, more weight will be given to measures the hospital did report. "We are still focusing on putting like organizations together, and there has been little to no focus on the underlying measures," she said.

Nerenz echoed that the quality measures used are still short of what is needed to provide an accurate star rating. For patients using Hospital Compare to decide where to get a joint-replacement procedure, the overall star rating won't show performance data for that individual procedure.

"If the whole idea is to promote informed consumer choice, what you really want to do is go down to the most granular level you possibly can and let consumers find the measures they are interested in," he said.

A possible solution the CMS said it's considering is to provide an individual star rating for each of the seven quality measure groups for each hospital.

Numerof said "being able to look at the underlying components that led to the measure is important," especially because CMS weighs the quality measure groups differently based on what the hospital chose to report.

Indeed, hospitals' performance on the seven quality measure groups shows wide variability. About 43% of hospitals performed above the national average on patient safety, while 34.5% of hospitals performed below the average for the measure. Performance on preventing readmissions was variable as well. About 40% of hospitals performed above the national average on readmissions, while 25% were about the same as the national average and 35% performed below the average.

Of the 4,578 hospitals on Hospital Compare, 3,692 met the reporting criteria for a star rating, the CMS said.

"We continue to refine the star ratings and look forward to an ongoing dialogue with hospitals and patients and their families on how we can provide beneficiaries useful information," CMS Administrator Seema Verma said in a statement.