CMS underestimated Medicare ACOs' savings

The Medicare Shared Savings Program generated $1.84 billion in savings over three years, which is nearly twice the savings that CMS data show, according to a new study commissioned by the National Association of ACOs.

The study, published Tuesday, also found accountable care organizations reduced Medicare spending by $542 million after accounting for shared savings by the ACOs from 2013 to 2015. This contradicts data from the CMS that ACOs actually increased Medicare spending by $344 million over the time period.

The study is the latest in a series that find Medicare ACOs are saving more money than CMS' methodology shows. Just last week the New England Journal of Medicine published an analysis with similar results. ACO analysts argue the CMS consistently underestimates savings because it's using a benchmarking methodology, which only compares ACO performance over the years to calculate savings.

Researchers argue the best way to analyze ACO savings is to compare their spending to Medicare beneficiaries not in ACOs, also called a difference-in-differences regression analysis.

The CMS likely doesn't conduct such analyses for the Medicare Shared Savings Program because it's not required to, said David Muhlestein, chief research officer at Leavitt Partners. Difference-in-differences analyses have been conducted for the Next Generation ACO and Pioneer ACO programs because they both originated from the CMS Innovation Center and are therefore mandated, he added.

But the lack of analysis by the CMS has caused some confusion about how much money Medicare ACOs are actually saving the agency.

"I think there is a growing understanding that you can't simply look at performance against the benchmarks as the definitive indication of how much savings are being generated and there are now academic studies finding that the magnitude of the savings tends to be higher," said Dr. Farzad Mostashari, CEO and co-founder of the ACO consultancy Aledade.

There are also greater stakes to understanding how much Medicare ACOs are saving as the CMS proposes to force more ACOs to take on risk faster, arguing the program isn't bringing down spending quickly enough. Many ACOs say they will leave the program if they are forced to take on risk before they are ready.

In the most recent study, conducted by consultancy Dobson DaVanzo & Associates on behalf of the National Association of ACOs, the spending by ACO-attributed Medicare beneficiaries was compared to non-ACO Medicare beneficiaries. The study uses claims data for 100% of ACO-attributed beneficiaries and 83% to 94% of non-ACO beneficiaries depending on the performance year.

The results were slightly different than previous studies that also use a difference-in-differences estimation methodology to calculate Medicare ACO savings, but Muhlestein said that is to be expected since researchers pick different characteristics to establish the control group, or group of non-ACO beneficiaries. The studies often conclude Medicare ACO savings are greater than estimated by the CMS.

"We hope that the study is going to change the narrative, and the narrative out there has been ACOs that don't take on risk don't save money, in fact they probably lose money. And that is not true," said Robert Mechanic, executive director of the Institute for Accountable Care who consulted with Dobson DaVanzo & Associates on the methodology for the study.

Muhlestein said that although ACOs are saving money, the savings "are still very small, and they are not bending the cost curve." Using the findings from Dobson and taking into account the overall spending on Medicare ACO beneficiaries over the three-year study period, the Medicare ACOs only generated a net savings of 0.3%, he said.

But Mostashari argues that even though the savings are small they are still savings. "What is the alternative again? What program saves more? Because I'm still looking for it."