The director of the Congressional Budget Office has rallied to the defense of his troops, one current and one former CBO researcher, who in a report last month criticized two older and oft-cited studies touting the macroeconomic benefits of widespread, national deployment of healthcare information technology systems.
Peter Orszag, director of the congressional watchdog agency, took to his blog earlier this month to answer criticisms of the CBO report sent to him in a letter and an attachment from officials at the RAND Corp.
To paraphrase, the RAND folks wrote to the CBO people saying, "You done us wrong," and the CBO boss blogged back, answering, "We stand by our men."
"Nothing in the RAND letter would cause us to modify our previous conclusions," Orszag wrote on his blog.
"The RAND study estimated potential savings of approximately $80 billion per year from health IT if it were widely adopted," Orszag continued. As the "CBO concluded in its recent report, however, that $80 billion figure is not an appropriate guide to the effects of legislative proposals aimed at increasing the use of health IT for several reasons. For example, the RAND study attempted to measure the potential impact of the widespread adoption of health ITassuming the occurrence of 'appropriate changes in healthcare'rather than the likely impact, which would take account of factors that might impede its effective use.
"In addition, the RAND study was based solely on empirical studies from the literature that found positive effects for the implementation of health IT systems; it excluded studies of health IT that failed to find favorable results," he wrote.
The CBO report was prepared by Stuart Hagen of its Health and Human Resources Division and Peter Richmond, formerly with the CBO.
Tasked by Congress with examining the macroeconomic impact of a major taxpayer investment in IT that is called for in several pending IT booster bills, the CBO researcher found that neither the RAND study nor another one focused on IT in ambulatory care provided "an appropriate guide to estimating the effects of legislative proposals aimed at boosting the use of health IT."
The RAND letter, dated May 27, was addressed to Orszag and was co-signed by Richard Hillestad, a principal researcher at the Santa Monica, Calif.-based think tank and the lead author of the RAND report.
Portions of the RAND report were first published in the September/October 2005 issue of Health Affairs. RAND itself also published additional information about its research. Physician Robert Brook, vice president and director of the healthcare section at RAND, was the other signatory of the complaint letter to the CBO.
Hillestad and Brook wrote in their letter to Orszag that their study "yielded an estimate of savings that could result from the successful future adoption of health information technology by 90% of the nation's hospitals and doctors. It is apparent that the CBO report set out to answer a different question with different assumptions. It is no surprise then that we might reach different conclusions. What is surprising and merits correction are the errors in omission and interpretation within the CBO report associated with criticisms of the RAND study."
The two spent a good portion of their letter defending the integrity of their process, listing a host of healthcare luminaries who served as steering-committee members who reviewed the researchers' work. They said their work "was carried out with total transparency. All evidence, methods and assumptions used in the study were made publicly available to allow for checking of the results."
The 2005 RAND study, whose numbers have been oft-quoted since 2005, has shown enough legs to make it into the presidential campaign as recently as a few months ago in a citation by Sen. Hillary Rodham Clinton (D-N.Y).
The RAND report estimated that the U.S. could save an average of $41.8 billion a year in healthcare costs if it broadly adopts (90% of hospitals and physicians) electronic health-record systems over a 15-year rollout period. Savings would climb to $77.4 billion a year when that 90% adoption mark is reached. The tab? An estimated $98 billion for hospitals and $17.2 billion for physician offices.
The RAND research project was funded by Cerner Corp., General Electric Co., Hewlett-Packard Co., Johnson & Johnson Services and Xerox Corp.
The CBO took pokes at both the RAND analysis and another well-quoted study by the Center for Information Technology Leadership at Partners HealthCare System, Boston. The CITL researchers estimated the nation would save $44 billion a year if it invested $14 billion to $30 billion on highly functional IT systems for office-based physicians.
Despite the popularity of the RAND and CITL reports, the CBO said, "For a number of reasons they are not an appropriate guide to estimating the effects of legislative proposals aimed at boosting the use of health IT."
The CBO researchers conceded that some well-known, integrated delivery systems such as Geisinger Health System, Intermountain Healthcare and Partners HealthCare System and the Veterans Affairs Department were getting a good bang for their bucks invested in IT, but for others, the returns were less sure.
"For providers and hospitals that are not part of integrated systems, the benefits of health IT are not as easy to capture, and perhaps not coincidentally, those physicians and facilities have adopted EHRs at a much slower rate," the report said. "Office-based physicians in particular may see no benefit if they purchase such a productand may even suffer financial harm."
Hillestad and Brook defended their work and took a couple of shots of their own at what they pointed out as inconsistencies in the CBO work.
"We stand by the methods and conclusions of the 2005 RAND study, and disagree with the CBO report conclusion that it overestimates the potential cost savings from health information technology," they said. "We particularly take issue with the effort to level that criticism without providing a robust alternative measure of potential savings that reflects CBO assumptions.
"In several sections of the CBO report, findings from the RAND study are accepted and integrated with the CBO analysis," Hillestad and Brook wrote. "Those areas of agreement with the RAND study are not reflected with citations. That might not be notable were it not for the specificity of the criticisms levied against the RAND study elsewhere in the CBO report."
In a parting shot, the RAND defenders noted that Hillestad was contacted by the CBO and asked to review and comment on only a small section of the report.
"Several comments on that material were returned to CBO but none was acknowledged in the final report," the RAND pair wrote. "Under these circumstances, it is most surprising to find Dr. Hillestad listed as a peer reviewer of the report."
What do you think? Write us with your comments at [email protected]. Please include your name, title and hometown.