On about a dozen measures, there was little difference between ACOs that earned bonuses and those that did not, including avoidable admissions for congestive heart failure, chronic obstructive pulmonary disease and asthma and readmission rates.
Still, ACOs that earned bonuses generally outperformed the rest on another 15 measures, including screening patients for risk of falls and reconciling medication for patients who leave the hospital.
Those that failed to earn bonuses, however, did better on a few measures, including timely access to care and the percentage of doctors who met the federal criteria for the meaningful use of electronic health records.
Avalere Health compared the quality results for the first year against the benchmarks that ACOs must meet in year two, when their bonuses will reflect points awarded for their actual quality performance.
More than half of the ACOs that earned financial bonuses scored below the 74-point average for all of the participants, a potentially worrying result.
Many of the participants earned the maximum points possible on the measure for all-cause readmission rates. “That's a great sign,” said Eric Hammelman, Avalere's vice president. But many earned zero points on avoidable admissions for heart failure, asthma and chronic obstructive pulmonary disease. “They're keeping people out the second time,” but not the first, said Hammelman.
Quality experts, however, cautioned that the data are not very useful on their own. Without additional information, the results cannot be compared against the providers' performance in the years before. That prevents researchers from using the data to explore whether ACOs saw more significant quality improvement than other hospitals and doctors.
“I view these much more as a comment about the quality of care provided than an assessment of the impact of ACOs,” said Dr. Michael Chernew, a health policy professor at Harvard Medical School who studies health spending and quality.
The limited data, however, may be enough to suggest that ACOs did not sacrifice quality to achieve savings. “I think that data like this allays those concerns,” he said. “They're not precise enough to be sure. It doesn't seem like there's a big disaster.”
Chernew said he believes the ACOs do in fact improve quality. “Many of the ACOs are trying to focus on chronic conditions and manage those populations where they tend to spend a lot of money,” he said. “That focus tends to do good things for quality.”
But others were hesitant to rely on the results even as a limited measure of quality.
“There's just not enough information to say anything definitive,” said Dr. J. Michael McWilliams, an associate professor of health policy at Harvard. It's not clear how quality scores account for different incidence of illness across communities, McWilliams said. ACOs with higher numbers of complex patients may find quality scores more difficult to achieve than ACOs in healthier communities.
McWilliams said ACOs vary in their ability to compile and report quality data, and more experience and additional data will yield a clearer picture of performance. Some of the measures, he added, are “probably not that good,” and performance on the measures can have little bearing on one another. “We just have to be cautious interpreting the numbers now,” McWilliams said. “This is the first time it's been done.”
And he described the fact that the vast majority of the participants successfully reported on the program's measures “an impressive early achievement on the part of the ACOs and CMS.”
But Dr. Ashish Jha, a health policy professor at Harvard, said the quality measures “are just not very good.” Too few of them assess the management of population health or the results that patients see from their treatment, he said.
“We're not measuring the things that matter most,” Jha said. “Are patients better off in an ACO? Are they healthier?”
Follow Melanie Evans on Twitter: @MHmevans
