Innovations: Facial recognition software measures pain

The experience of pain is inherently subjective. One patient's rating of pain as 5 on a 10-point scale could be another patient's 10.

It's difficult enough for providers to assess pain levels in adult patients with the ability to describe their pain and compare it with past experiences. It's much harder in treating children or adults with dementia, who often are not able to verbally describe their pain. Now some researchers are seeking a technological way to objectively measure pain.

Dr. Jeannie Huang, a pediatric gastroenterologist at Rady Children's Hospital-San Diego, said she finds it distressing not being able to accurately assess her young patients' pain. Some of her patients have stayed in the hospital longer because clinicians did not accurately assess their pain, identify the source and treat the problem effectively. Huang wants to find an unbiased way to assess pain in patients who can't tell her.

A 2008 study in the Journal of Advanced Nursing found that nurses' estimates of pediatric patients' pain were only moderately correlated with patients' self-reported pain rating.

“The bottom line is (that) what the patient is experiencing is what's most important,” said Bob Twillman, executive director of the American Academy of Pain Management. “To the extent that we're able to elicit that determines whether we'll be able to give (patients) the relief they need.”

Huang, a professor of pediatrics at the University of California, San Diego, solicited the help of Marian Bartlett, a research professor at UCSD's Institute for Neural Computation. Bartlett was using facial recognition software to teach autistic children how to recognize and interpret facial expressions. Huang wondered whether the same software could be used to measure pain.

In 2012, Huang, Bartlett and a team of researchers launched a five-year National Institutes of Health-funded project to determine whether pediatric patients' pain could be accurately measured by facial recognition software developed by Emotient, a San Diego company co-founded by Bartlett.

The prototype software was programmed to recognize 20 facial muscle movements known to indicate pain. The application is based on data from prior software developed by Emotient that uses computer vision techniques to analyze facial expressions.

Through a process of machine learning, the software is “taught” to measure pain based on video images of patients in pain. Then it performs a regression analysis of the levels of intensity in those patients' displayed signs of pain. The software applies the data to measure pain in other subjects, using a 0-to-10 scale.

In a study to be published in the July 2015 issue of Pediatrics, the software was taught and tested on 50 children, ages 5 to 18, who had undergone laparoscopic appendectomies at Rady Children's Hospital. Video images of the children's faces were captured within 24 hours after their appendectomy, one day later, and at a follow-up visit two to four weeks after surgery.

The application's pain ratings correlated strongly with patients' self-reported pain levels. The software's assessment of pain came closer than nurses' assessments to those self-reported levels. And the software performed as well as the patients' parents in estimating pain severity. That's significant because parents' estimates have been shown to be closer than clinicians' estimates to children's self-reported pain.

Bartlett said Emotient eventually hopes to market the pain recognition software. It's not clear whether the device would require Food and Drug Administration approval. The company still needs to refine the software using more video of patients experiencing various levels of pain.

Dr. Lynn Webster, a past president of the American Academy of Pain Medicine, said there's promise in the Emotient application's real-time ability to help clinicians assess pain and better calibrate their medical interventions. It could help physicians reduce or increase pain medications. “We're always seeking biomarkers that can help us treat chronic illnesses like pain,” Webster said. “Facial recognition could be very helpful in providing useful information.”

Bartlett said that enabling providers to track patients' pain is what could make the Emotient application commercially viable. The company is in talks with several telemedicine providers interested in the software and would like to bring it to market in 2016, she said.