Local news sites have been quick in the past week to tout ratings, newly released by the government, of hospitals in their neighborhoods. “Gulf Breeze rated top local hospital under new system,” read one headline in the Pensacola (Fla.) News Journal. “14 of 20 local hospitals win 4 stars in US quality ratings,” trumpeted another, on Cincinnati.com.
Others sounded less triumphant. “Local hospitals rank low in new CMS rating system,” the St. Louis Business Journal noted. But whether hospitals score poorly or highly, the question that remains is how the ratings, which have drawn much attention and stirred controversy within the industry and on Capitol Hill, will affect patients.
Advocates of the star-rating system say it will fuel more-informed decisionmaking, allowing consumers to select the best hospital for their needs. Critics argue that patients are dealing with inaccurate information that might lead them to choose the wrong hospital. There's also the fact that, depending on a patient's ZIP code or medical situation, they might not have the option of picking a hospital.
“I honestly don't know how much the American public will be able to use these ratings in the current form,” said Dr. David Levine, medical director and vice president for advanced analytics/informatics at Vizient, a healthcare performance improvement company based in Texas. “We're very supportive of CMS' overall goal,” Levine added, referring to the use of star ratings to better inform patients. But there's a lot of room for improvement with the data—which could be more timely or adjusted for hospitals that tend to serve sicker patients—to be sufficiently applicable, he said.
The CMS released its ratings last week, following a last-ditch effort by industry stakeholders pushing for postponement. The ratings, a composite metric of one to five stars, with five being the best, were assigned to 3,617 hospitals and posted on Medicare's Hospital Compare website.
Industry stakeholders have vociferously questioned the precision and accuracy of the metrics and the adequacy of CMS' methodology. They have called for better risk-adjustment so that hospitals treating disproportionate numbers of poor or uninsured patients, who have less access to healthcare in their communities once they leave the hospital, aren't penalized.
New York state hospitals tend to treat higher percentages of Medicaid or uninsured patients than hospitals in other states, so they tend to be chronically underfunded, said Karen Heller, executive vice president at the Greater New York Hospital Association, which opposed the release of the star ratings. As a result, their facilities are less well-appointed, a factor that will likely drive down patient satisfaction and therefore overall scores. New York state law says hospitals can't turn away patients in emergencies even if they don't have a designated emergency department.
“There's no allowance for the fact that a hospital is going to be underresourced relative to other ones,” Heller said. “If you don't have an emergency room, you can pick and choose your patients,” she added.
Not only was it “unfair” to hospitals to be characterized as “bad,” Heller said, but they then also were burdened with explaining a low star rating to the media and to their trustees.
To determine hospitals' scores, CMS took 64 measures that hospitals were already publicly reporting and categorized them into seven broad groups: mortality, safety of care, readmissions, patient experience, effectiveness of care, timeliness of care and efficient use of medical imaging.
It used a statistical modeling approach called latent variable modeling, which is supposed to accommodate variations in the amounts of data and information hospitals report, to develop a single score for each group, it explained in a methodology report.
Then, to create the final, composite score for each hospital, each group received a different weight. Mortality, safety of care, readmissions and patient experience each constituted 22% of the overall score, while effectiveness of care, timeliness of care and efficient use of medical imaging were worth 4% each.
In order to receive an overall star rating, hospitals had to report at least three measures each in at least three groups, at least one of which had to be an outcomes group (mortality, safety of care and readmissions). So, a hospital could potentially not report data in four groups yet still receive an overall score.
For hospitals that did not report measures in a given group, CMS re-proportioned the weight of the missing group across the groups that were reported. For example, if a hospital did not report any efficient use of medical imaging measures, worth 4%, each of the other measures were recalculated to be worth slightly more, out of 96 instead of 100.
The CMS said in explaining its methodology that star ratings “do not reflect an 'apples to apples' comparison between hospitals.” A hospital might receive three stars because it does extremely well in some groups and poorly in others. Another hospital that ranks as average across them all, however, would also receive three stars.
Consumers should be expected to understand those nuances, said Leah Binder, CEO of the Leapfrog Group, which publishes its own grades for hospitals. “I don't think consumers are quickly confused.”
The ratings would prove useful to patients, Binder said, because “people do not want to have to plow through endless reams of data to figure out which provider is best for them. They appreciate ratings that offer a singular assessment,” precisely what the star ratings are.
Whether patients will have more than one hospital to choose from, whether due to geography or insurance coverage, is another matter. Hospital density varies by state—in 2014, Oregon had 59 hospitals, while Oklahoma, which has a similar-size population but is a roughly a third smaller physically, had nearly twice that, according to the Kaiser Family Foundation—as does the number of hospital beds per 1,000 people.
In 2014, the states of Washington and Oregon had the fewest number of hospital beds—1.7—per 1,000 people. South Dakota, by comparison, had 4.8 beds per 1,000 people.
In emergencies, patients also might not have a choice in which hospital they end up in. “We always encourage consumers to consult ratings long before they think they'll need a hospital,” Binder noted. She added that the new CMS ratings could be used as one of many sources for patients in judging hospitals. “It's not the only tool,” she said.
Still, some say that the government should alert customers to what exactly they're getting when they use that tool, even as CMS works to improve the quality of the ratings now that they are public.
“I would hope that CMS would try to put some appropriate caveats on their site with these ratings, to help inform the public that this is just one element of evaluation,” Vizient's Levine said. “We wouldn't want consumers to be making decisions based on incomplete information.”
