CMS, Yale New Haven Health on hot seat over design of quality measures

The CMS' long-standing hospital readmissions penalty program has taken quite a beating recently from members of the research community over problems with its underlying measures.

But while criticism of the program has escalated, not everyone has joined the bandwagon. Dr. Harlan Krumholz, a well-known researcher at Yale New Haven Health, passionately defends the measures, and with good reason: He led the group that helped the CMS develop the readmission measures as part of a multimillion-dollar contract the agency gave to the academic healthcare institution.

Still, dissatisfaction with the way the readmissions program is designed is coming to a head, and possibly has exposed some fundamental flaws with not just the readmissions program but the CMS' measure-designing process, including those found on Hospital Compare and part of the CMS' value-based purchasing programs.

“I think the measures we end up with are a function of the process we use for the development of these measures … and none of the processes that we use are really particularly good,” said Francois de Brantes, senior vice president of commercial business development at healthcare consultancy Remedy Partners who has experience bidding for CMS contracts.

Drawing attention to the CMS' Hospital Readmissions Reduction Program in the past month were two studies published in different journals. One—published in late December in JAMA—found mortality rates rose slightly among heart failure and pneumonia patients after the readmissions program went into effect, mirroring another study published in November 2017 that came to similar conclusions.

The other recent study—published this month in Health Affairs—found that reductions in readmissions rates may be related to coding changes the CMS made around the time the program was implemented. This was the second study in the past year to suggest that the reductions in readmissions may have been overestimated.

The research has raised concerns across the healthcare industry, and a CMS spokeswoman said it's “thoroughly” reviewing both the recent studies to “inform any future actions.”

“Any finding in the medical literature suggesting that CMS programs may be associated with increased mortality is of deep concern, and we take the safety of our patients very seriously,” the spokeswoman added.

But the studies don't dissuade Krumholz from defending the program. When the JAMA piece was published in late December finding a correlation between rising mortality rates and the readmissions program, Krumholz published a Twitter thread knocking it down with some of his own studies. One of those, published last July in JAMA, found that among hospitals that reduced 30-day readmission rates, 30-day mortality rates after discharge actually decreased slightly. The first author of the study, Dr. Kumar Dharmarajan, is also affiliated with Yale. JAMA discloses in the study that the authors work under contract with the CMS but doesn't say Yale New Haven was integral to developing the CMS' readmission measures.

Krumholz and Yale have a decadelong history with the CMS. Krumholz led the development of the risk-adjustment model the agency uses for the readmission measures and his employer has received about $135 million from the CMS since 2009 to develop and maintain the readmission measures, along with several other quality measures embedded in agency programs. Krumholz admitted his ties to the CMS in his December Twitter thread but the research community is largely silent on his interest, possibly out of fear of rebuke.

“Nobody in academia was willing to bring up the fact that Harlan might have some conflicts—there are reasons he is so invested in the program,” said Dr. Anish Koka, a cardiologist in private practice who recently wrote a blog post for the Accad & Koka Report about funding Yale has received from the CMS through contracting.

“I don't think it is a secret that I am … involved in the design of the readmission measures and that I work with CMS to build tools to support policies to improve healthcare,” Krumholz said in an email exchange with Modern Healthcare. “CMS has never asked me or anyone on my team to defend the readmission policy.”

Health policy analysts who have personally experienced or witnessed how the CMS contracts with groups like Yale to develop quality measures say the recent questions raised about the readmissions program demonstrate the shortcomings deeply embedded in the CMS' contracting process for quality measure development. Currently, the CMS' quality measures rely heavily on expertise and research from a select handful of organizations that receive millions of dollars to do so.

As a result, the industry uses measures that lack diversity and rigor, the analysts said. “Ideally, we would have more competition and innovation in measure development,” said Dr. Karen Joynt Maddox, an assistant professor of medicine at the Washington University School of Medicine in St. Louis who has studied the readmissions program. “By having the same contractors year upon year, it means we are locked in to how they want to do things.”

The CMS uses organizations to develop and maintain quality measures through a contract called Measure and Instrument Development and Support, known as MIDS, and a review of public documents shows that the CMS contracts with the same groups.

Yale New Haven Health and 17 other organizations currently have a $800 million contract with the CMS to develop quality measures. Yale has received about $110 million from the contract since 2013 and the funding expires this year. Yale New Haven is the biggest awardee of the MIDS contract for 2013, and the institution is responsible for a plethora of hospital quality measures in use today.

Along with its work on all of the readmission measures, Yale New Haven has developed the methodology for the overall hospital star ratings, which has been panned by quality researchers and hospital stakeholders, and for many of the measures on Hospital Compare, like acute myocardial infarction mortality, pneumonia mortality and elective hip or knee replacement complications. The work typically comes out of the Center for Outcomes Research and Evaluation, which is affiliated with the Yale School of Medicine and Yale New Haven Hospital; Krumholz is the center's director.

In August, the CMS announced that Yale New Haven was selected with 30 other organizations for another contract to develop quality measures, this time with a funding cap to the groups of $1.6 billion. Of the 18 awardees from the 2013 MIDS contract, 16 were awarded again for the 2018 MIDS contract.

A CMS official said in an interview with Modern Healthcare that the agency hasn't intentionally selected a relatively small number of organizations for its MIDS contracts. Rather, the pool of organizations who can do the work is not very large.

“We want to ensure that whoever we are contracting with either themselves or they have access to subcontractors who have the requirements, expertise and experience. These measures that are being developed are being used for payment on a national scale,” she said. “I believe our standards for what we are looking for are rigorous, and they ought to be given what is at stake. … There are not hundreds and hundreds of organizations that have that” expertise.

The CMS official noted that the number of organizations with measurement development expertise is growing, which is reflected in the increase in organizations the CMS awarded to be part of its most recent MIDS contract.

“There are more people getting into this space and developing the experience because measurement has become one of those tools not just used by us, but commercial payers too,” she said.

But a Modern Healthcare review of the 2013 MIDS contract shows Yale, Mathematica Policy Research and Health Services Advisory Group were by far the largest awardees of CMS' $800 million funding pool, while two organizations didn't get any funding at all for projects.

RELATED STORY: How the CMS picks winners of valuable quality measure contracts