Payers and Purchasers: Numbers that count

For more than a decade, the disease-management industry’s approach to evaluating and reporting outcomes has been about as comprehensible as baseball legend Casey Stengel famously directing a group of rookie players to “line up alphabetically according to height.”

Since the industry’s inception in the mid-’90s, disease-management vendors have assessed their clinical and financial outcomes using a hodgepodge of competing and often flawed methods, yielding results that even the staunchest advocates admit have strained plausibility and defied comparison across programs.

But now the Disease Management Association of America is trying to get everyone on the same page. In December 2006, the Washington-based trade group released an initial set of consensus guidelines designed to help standardize the way disease-management programs measure and report their outcomes. And this year—in what it calls Phase 2 of its outcomes project—the DMAA plans to refine those guidelines and supplement them with a set of disease-specific clinical metrics, which could be incorporated into a formalized accreditation program down the road.

“It became clear not only to our membership, but also to the purchasers of disease-management services … that there needed be some recognized, standardized set of guidelines to alleviate the significant amount of confusion that had arisen around the true impact of these programs as a direct result of the numerous methodologies that were out there,” says Tracey Moorhead, the DMAA’s president and chief executive officer.

The guidelines are the result of a yearlong effort by the DMAA that incorporated recommendations from association members, as well as a broad array of external stakeholders, including employers, benefits consultants, academics, actuaries, government agencies and accrediting bodies.

The effort comes in response to the fallout of a damaging report released in October 2004 by the Congressional Budget Office, which concluded that there was “insufficient evidence” to prove that disease-management programs actually save money. Since then, disease-management vendors have been scrambling to build a better business case for their services, largely at the urging of employers and health plans that have begun demanding greater proof that the millions of dollars they are investing in chronic-care programs are being put to good use (Aug. 8, 2005, p. 26).

But to what extent the guidelines will actually solve the industry’s credibility problem remains to be seen. Some observers question whether the voluntary nature of the guidelines will pack enough punch to prompt vendors to change their established way of doing things. Others point out that the guidelines equivocate on a number of key issues, preventing true standardization even if they were adopted industrywide.

“If the goal is to have everyone speaking the same language … I think there are several controversial areas that will need to be revisited,” says Michael Mustille, associate executive director of external relations for the Permanente Group, the physician arm of Kaiser Permanente.

By most accounts, the guidelines are a good first step, tackling some key issues that have long been a source of industry debate and public confusion. For example, the DMAA recommends that disease-management vendors calculate their outcomes in terms of total-dollar savings rather than return on investment, a much more fluid ratio that critics say can muddy financial comparisons, and is often used to exaggerate results.

“They took a stand on a lot of issues where they could have punted and done nothing or just made broad pronouncements, ” says Alfred Lewis, founder and president of the Disease Management Purchasing Consortium International, a Wellesley, Mass.-based organization that provides contracting assistance to health plans and other purchasers of disease-management services. “I think they did an absolutely great job for year one. There’s no denying that.”

However, Lewis says the guidelines have two major holes: They don’t address key statistical biases that can skew results, and they don’t call for a plausibility test, which confirms outcomes by applying a second evaluation methodology. Even so, he says the DMPC plans to incorporate the guidelines into the contracts and requests for proposals the consortium writes for its members.

Other industry observers say the guidelines leave too much wiggle room in certain areas, such as study design. For example, after much debate among its members, the DMAA decided to endorse two competing approaches to identifying patients for the purpose of program evaluation, “annual qualification” and “prospective identification.” Because the two methods differ in the way they define the population to be measured, a company using annual qualification could generate very different outcomes over time compared with a company using prospective identification.

“What these guidelines do not do is provide the level of detail and specification that somebody could follow to produce results that could truly be comparable from one organization to another,” says Joachim Roski, vice president of performance measurement for the National Committee for Quality Assurance.

The DMAA admits that it took a “middle of the road” approach to the guidelines in Phase 1, often making concessions to achieve maximum consensus. It also opted to sacrifice some scientific precision for the sake of practicality, says Gordon Norman, executive vice president and chief medical officer for Reno, Nev.-based disease-management firm Alere Medical, who sat on the DMAA’s 45-member outcomes steering committee.

For instance, the most scientifically valid outcomes studies typically involve a randomized, controlled clinical trial. But because such trials are difficult to conduct in real-world settings, the DMAA’s guidelines recommend the use of a “quasi-experimental” study design that incorporates “an equivalent, concurrent comparison group.” Yet even then, the DMAA tempers its stand, adding that if a suitable comparison group isn’t available, companies should make “explicit efforts” to control for potential biases and errors introduced by the study design.

“Did we lose credibility by waffling? Possibly,” Norman says. “But frankly, I think the world of DM is comprised of enough diverse circumstances, vastly different populations, different programs and different outcomes of interest to the sponsors of these programs that to have tried to distill that heterogeneous reality to a single approach would have been premature and foolhardy, possibly jeopardizing even greater credibility.”

“I think we’re well on the way to espousing a set of acceptable principles and, believe me, there were many, many people functioning outside those principles previously,” Norman adds.

DMAA officials emphasize that the guidelines are still a work in progress. Over the next year or two, the association plans to tighten up its existing recommendations and to “add more meat” to them by developing a list of statistical biases and ways to reduce their impact on outcomes, says Jeanette May, the DMAA’s vice president of research and quality. “I believe you’ll see the guidelines become more specific in Phase 2,” she says.

In the meantime, it will be up to the disease-management companies themselves to determine how to make use of the guidelines.

Cigna Corp., which operates several disease-management programs through contracts with vendor Healthways, says it plans to adhere to the DMAA’s recommendations. “Cigna is absolutely looking at the guidelines in informing the way we bring results to the marketplace,” says Christopher Coloian, Cigna’s vice president of health advocacy products, and one of four Cigna executives who sat on the DMAA’s outcomes steering committee. “Really, Cigna was already practicing the majority of the guidelines … and I think we had strong input in the process.”

But Kaiser Permanente, the nation’s largest not-for-profit HMO with 8.2 million members, has no plans to adopt the guidelines. As an integrated delivery system that does all of its disease management in-house, the company has already established “a more complete way of knowing what our costs are and what our medical impacts are than other disease-management companies,” says Mustille, who also sat on the outcomes steering committee.

“We wouldn’t use this kind of financial outcomes methodology as the sole evaluative tool for our programs internally,” Mustille says. “But I do think it makes sense for those that don’t have an integrated program like ours. So while these guidelines aren’t going to be particularly useful for us, I do think they’re a good first step in the process of building credibility in the marketplace.”

In addition, Lewis believes the guidelines are unlikely to have much impact on vendors that are far smaller or new to the disease-management market, which has grown at a 40% compound annual growth rate since 1997, to nearly $1.2 billion.

Many upstart companies may be reluctant to adopt more stringent or standardized evaluation methods because doing so would jeopardize the higher return-on-investment figures they’ve relied on to win clients in a market where more than 100 companies compete for business. Others, Lewis says, simply don’t have the experience to recognize their evaluation methods aren’t up to snuff.

“What the guidelines are going to do is encourage the people who are already pretty much in compliance to get slightly more in compliance,” Lewis says. “But there’s a whole unwashed mass of disease-management companies that fly under the radar screen, and these guidelines aren’t going to help them at all. You have to know that you have a problem to change the problem, and there are a lot of people out there who simply don’t have a clue.”

Greater consensus won’t be achieved, Lewis says, until more purchasers begin to challenge ambiguous, contradictory and exaggerated outcomes. “I am just amazed by what employers will still accept,” he says.

Dennis White, senior vice president of value-based purchasing for the National Business Coalition on Health in Washington, agrees that employers will have to play a bigger role in driving industry change. “I think it will take the purchaser community and (other outside) organizations to step up and start using these guidelines and to demand their use,” he says, admitting that “there’s still a lot of naivete among employers.”

But White contends that for the guidelines to have real teeth, there needs to be a third-party organization in place to collect, audit and publicly report disease-management vendors’ performance and compliance data, much the way the NCQA provides HEDIS scores for health plans or the CMS rates the performance of hospitals that participate in the Medicare program on its Hospital Compare Web site, hospitalcompare.hhs.gov.

The guidelines “are a very important first step, outlining some good principles and getting things on the road to filling some of the measurement gaps,” White says. “But until there’s an infrastructure in place to actually track the use of those metrics in a neutral way, they’re not that useful.”

The NCQA is already considering taking up the mantle. In December, the organization began collaborating with the DMAA to develop a set of disease-specific clinical performance measures, which it plans to incorporate into its existing disease-management accreditation program. Its current program, launched in December 2001, evaluates vendors on their operations, clinical systems and service, but not their outcomes.

“To our frustration, we had not been able to integrate performance measures into our program, largely because the industry was not ready for that and because of various data-collection concerns,” the NCQA’s Roski says. “But I think that we have now reached a point where the industry is mature enough for us to introduce metrics that will ultimately drive public reporting and allow one to compare vendors on an apples-to-apples basis for the first time.”

The DMAA plans to release the new metrics—which will measure clinical outcomes, as well as quality, productivity and utilization—at its annual conference in September. The NCQA hopes to fold the metrics into its accreditation program by late 2008, Roski says.

Separately, the DMAA last month unveiled a new standardized patient-satisfaction survey for use by disease-management companies. The association plans to compile the survey results in a de-identified database that members will be able to tap for their internal quality-improvement efforts. The DMAA does not plan to publicly report the survey results, May says.

What do you think?