Talk about your good news-bad news situations.
Three recent pediatric medical journals have studied the implementation of computerized physician order entry at Children's Hospital of Pittsburgh and have linked the process to both a reduction in harmful medication errors and an increase in mortality.
Not surprisingly, it is the most recent report and the one with negative findings that has received the most attention.
A report in the December issue of Pediatrics says that mortality rates for patients transferred to Children's Hospital for specialized, tertiary-level care increased to 6.57% (36 out of 548 patients) for the five months after CPOE implementation, from 2.8% (39 out of 1,394) for the 13 months prior to implementation. (The study began Oct. 1, 2001, with implementation occurring near the end of October 2002. The study ended on March 31, 2003.)
These negative findings contrast with a report that appeared in the January issue of the Journal of Pediatric Surgery which found that transcription errors were eliminated and harmful adverse drug events dropped to 0.03 per 1,000 doses from 0.05 per 1,000 doses. This equals the prevention of one harmful adverse drug event every 64 patient days. (Children's Hospital is a 260-bed facility with 12,000 annual admissions, including about 3,000 annual intensive-care unit admissions.)
A report in the November issue of Pediatrics described the planning and preparation that Children's Hospital underwent before implementation and concluded that "CPOE is an invaluable resource for supporting patient safety" and that "Our patients appreciate the improved safety and care that CPOE supports."
A representative from Children's Hospital was unavailable for comment.
On health information technology Web logs, however, the commentary is being generated fast and furiously.
"There has been a boatload of cyberspace discussion," Jim Fackler, M.D., a pediatric intensive-care physician and critical-care director for IT vendor Cerner Corp., whose Millennium system "went live" at Children's Hospital in October 2002.
"It is my belief that these are true-true and coincidental findings," Fackler said. "It's true CPOE went in. It's true mortality went up. It remains my opinion that the two aren't causally related."
The December Pediatrics report notes that there were several problems that may have led to the mortality increase.
There was a lack of communication bandwidth which sometimes "froze" the computers; in situations when patients needed to be stabilized, two physicians were often needed, with one directing care and the other working the computer; there was a lack of standard order sets for common pediatric ICU patient conditions; and a satellite ICU medication dispenser was removed in order to centrally locate all the hospital's pharmacy services.
"Nothing in the article points to it being the CPOE technology itself that was at fault," said Suzanne Delbanco, executive director of the Leapfrog Group, which has been a strong promoter of CPOE. "It was the implementation and surrounding infrastructure that caused the problems."
Fackler was also highly critical of the study's design, which compared 13 months prior to implementation with just five months after.
"It's just bad," he said, adding that pediatric mortality rates have been shown to have seasonal fluctuations and a valid study would have compared identical time periods.
The researchers appear to have anticipated this criticism and noted in their report that the study was cut off after five months because that was when the 80-hour limit for resident work hours was initiated. This was felt to be a "potential confounder" of the results so the study was terminated when the limit was imposed.
They added, however, that "distribution of diagnostic categories" were similar during the two periods studied.
Regardless of any shortcomings, the study will probably have some impact, said Shahid Shah, an IT consultant. Shah said the immediate effect may be to slow down hospital administrator decisions on CPOE, and it may have a more "disruptive" and long-term impact if it is determined that health IT requires some sort of oversight from the Food and Drug Administration.
The cyberspace discussions mentioned by Fackler may be where the study's real benefit exists, Shah said -- especially if it promotes more pre-implementation testing.
"The value of the study is that it starts a conversation," he explained. "It's a wake-up call to all health IT vendors that you need field studies and (workflow) simulations in place."
Shah noted that, "We don't put planes in the air without simulations," so it makes sense to do a simulation before implementing IT applications, since lives could be at stake in either instance.
Another reason the report is generating so much attention, Shah said, is that -- instead of studying something such as how many more patients a physician can see in an hour -- it measures something the public cares about: mortality rates.
Fackler added that Cerner doesn't abandon a facility once its systems are installed and that the company offers plenty of resources and guidance for implementing and operating its products.
"We have a substantial amount of stock content and an enormous amount of consulting expertise, and we make these available to any institution that so chooses," he said. "Ultimately, however, patient care and systems are the responsibility of the individual sites."