Vendors dispute EHR, ambulatory-care report

A report detailing how the use of electronic health records does not necessarily lead to an increase in the quality of care may be misinterpreted by some as proof that EHRs aren’t useful. EHR vendors, consequently, are concerned.

"It's caused quite a bit of discussion in our industry—to say the least," said Hugh Zettel, director of government and industry relations for GE Healthcare. "We don't believe the reporting on it has been accurate relative to the findings of that paper."

The report, Electronic Health Record Use and the Quality of Ambulatory Care in the United States, appeared in the July 9 edition of the Archives of Internal Medicine, and concluded quite bluntly that: "As implemented, EHRs were not associated with better quality ambulatory care."

Written by prominent health information technology figures from Harvard Medical School and Stanford University, the study examined records of 50,574 patient visits collected as part of the National Ambulatory Medical Care Survey in 2003 and 2004, and compared how physicians with and without EHRs did on 17 quality measures. The researchers concluded that EHR-using physicians had significantly better scores on only two quality indicators, had no significant difference on 14, and did significantly worse performance on one.

"The result was surprising," said the study's lead author, Jeffrey Linder, an assistant professor of medicine at Harvard Medical School and an internist at 746-bed Brigham and Women's Hospital, Boston. "I was expecting to find that it (EHR use) was associated with better care."

Linder said that most EHR quality studies have been done at what he described as "benchmark" institutions, and the intent of this study—which was sponsored by the Agency for Healthcare Research and Quality—was to take a more general view of how EHRs were being used across the nation. What the study shows, Linder said, is that with the way EHRs are being used they "are not much more than a replacement for the paper chart."

"They're not magic," Linder said. "You just can't plug it in, turn it on and watch quality magically improve."

The two measures that the EHR-using physicians scored significantly better involved avoiding prescribing benzodiazepine to patients with depression and avoiding unwarranted urinalysis testing. The authors were surprised to report that EHRs were associated with worse quality when it came to prescribing statins to treat hyperlipidemia, or high cholesterol.

Linder said that he spent two days in vain trying to figure out that result. "It could be just statistical chance ... it could be a statistical anomaly," he said. "I don't have a good explanation."

Zettel disputed some of the findings, saying that GE Healthcare's own research found that its customers had scores twice as high as those the researchers found on quality indicators relating to aspirin, beta blocker and statin prescribing. "We have a process that allows our customers to show these and other related metrics," he said.

Mostly, however, Zettel said the findings may be a reflection of when half the data were collected: 2003.

"A lot has changed since then," he said, and this includes an evolving definition of "EHR."

According to the report, about 16% of the visits studied from 2003 involved EHRs, as did 20% of the visits in 2004.

Another of the study's co-authors, Randall Stafford, an associate professor of medicine at Stanford University's Prevention Research Center, acknowledged Zettel's arguments, but said the findings point to the need for multidimensional solutions to confront the complex problems relating to healthcare quality. These include a need to look at how healthcare is organized and paid for and how continuity of care is provided for chronic conditions, he said.

"The bottom line is that people have to pay attention to more than just the EHR and to think that the electronic health record will improve quality on its own is ridiculous," Stafford said. "The electronic health record in and of itself is not going to be adequate."

Additionally, the report states that "it is worth noting that the performance on most indicators was suboptimal regardless of whether an EHR was used."

Zettel somewhat agreed with Stafford's assessment.

"There's that old axiom that a fool with a tool is still a fool," he said. "And, if you don't change your processes, (implementing technology) will just help you make the same mistakes faster and more efficiently."

The report's other authors are Stanford's Jun Ma, a research associate; Harvard's David Bates, a professor in the school's health policy and management department; and Blackford Middleton, the chairman of the Center for IT Leadership who is also the former chairman of the Healthcare Information and Management Systems Society. Linder said having those names on the report "makes the results more believable," and this may be why there hasn't been a more "visceral" negative reaction from the industry.

The authors' prominent place in the health IT community is significant, said Gordon Schiff, who is leaving his post as director of clinical quality research and improvement at Chicago's 460-bed John H. Stroger Jr. Hospital of Cook County to become an associate professor at Harvard Medical School.

"Have to commend the researchers for honestly facing up to the evidence they uncovered, even though they themselves are leading advocates for EHRs," Schiff wrote in an e-mail. "Rather than being cheerleaders for selling computer systems—as many of the vendors are—they are helping move us to the next phase and hard work of crafting systems that actually work better for physicians and patients."

Zettel said that he didn't think the study would have a significant, immediate economic impact, but he worried about the political fallout.

"Since we're in the midst of trying to drive adoption, it creates a speed bump," he said. "Particularly, with sources on the Hill who may read this and say 'Tell me again why I should try to incentivize adoption.' "

Maryland-based IT consultant Shahid Shah, who operates the "Healthcare IT Guy" Web log and is the chief executive officer of Netspective Communications, said that EHR use is currently so low it's a stretch to make any universal claims from such a small data pool. While it is a setback for some that the study did not offer definitive proof of health IT improving clinical outcomes, he said that type of proof will probably not materialize for some time.

"But it doesn't take away from the fact that it still has some operational efficiency, and it may help reduce errors through better computational capabilities," Shah argued.

Shah said that people are going to use the report one way or another to prove whatever point it is that they want to prove.

Linder said he was a little concerned about that.

"If you're an electronic health-record vendor, this is not helpful to you," he said. "I have a fear that this would be misrepresented as 'Electronic health records don't work.' I'm a fan of electronic health records, but you need that culture that's focused on quality and safety as well. This may help move that along."

What do you think? Write us with your comments at Please include your name, title and hometown.