Are computer programs that read text-based medical records ready for prime-time use in quality improvement? Maybe so, according to research published in the latest issue of the Journal of the American Medical Association.
Natural language processing better for spotting quality lapses after surgery: study
Quality-improvement researchers concluded that computerized natural language processing of free-text portions of patient medical records was more effective in identifying quality lapses in post-operative surgical patients than a computerized review of discrete data elements in those records. Natural language processing, or NLP, is the use of computers to read and process information expressed in human language.
The researchers looked at the randomly selected records of 2,974 hospitalized surgical patients at six U.S. Veterans Affairs Department medical centers from 1999 to 2006 that were reviewed through the Veterans Affairs Surgical Quality Improvement Program.
A report on their findings, "Automated Identification of Postoperative Complications Within an Electronic Medical Record Using Natural Language Processing," appears in the Aug. 24/31 issue of JAMA.
In conducting the study, researchers obtained from the VA's VistA electronic health-record system narrative clinical notes, such as discharge summaries, progress notes, operative notes, microbiology reports, imaging reports and outpatient visit notes.
The quality-improvement program records had been assessed for 20 "patient safety indicators" developed by the Agency for Healthcare Research and Quality that rely on structured administrative data, such as ICD-9 codes, from hospital discharge records to identify possible adverse events.
Nurse reviewers involved in the study tracked the cases for 30 days after surgery and recorded the occurrences of these post-operative complications.
Researchers focused on six events or conditions: acute renal failure requiring dialysis, sepsis, deep-vein thrombosis, pulmonary embolism, myocardial infarction and pneumonia. They compared the results from the two search methods—natural language processing and structured data query—for sensitivity, defined as "the proportion of the six postoperative events that were identified by either the natural language processing or the patient-safety indicator approach." Specificity was defined as the proportion of hospitalizations without targeted adverse events that were not flagged by the corresponding natural language processing or patient-safety indicators.
The researchers found that in general, using a natural language processing-based approach had higher sensitivities and lower specificities than did the patient-safety indicators.
"Natural language processing correctly identified 82% of acute renal-failure cases compared with 38% for patient-safety indicators," the researchers wrote. Similar results were seen for other events, though differences varied based on the condition, with the discrepancy highest for pneumonia (64% vs. 5%) and lowest for postoperative myocardial infarction (91% vs. 89%).
"The development of automated approaches, such as natural language processing, that extract specific medical concepts from textual medical documents that do not rely on discharge codes offers a powerful alternative to either unreliable administrative data or labor-intensive, expensive manual chart reviews," the study authors concluded. The research team was led by Dr. Harvey Murff of the Veterans Affairs Medical Center and Vanderbilt University, both in Nashville.
Send us a letter
Have an opinion about this story? Click here to submit a Letter to the Editor, and we may publish it in print.