Your doctor may be better than an online symptom checkers to diagnosing what's wrong with you, but he or she still misdiagnoses cases 15% of the time, according to a report published Monday in JAMA Internal Medicine.
Online symptom checkers are becoming more popular among consumers, so researchers at Harvard Medical School and Boston-based Brigham and Women's Hospital tested whether trained physicians or online symptom checkers were more likely to accurately diagnose a clinical case.
The researchers used 45 standardized patient cases to test the accuracy of 234 physicians against 23 symptom checkers, which are websites and apps that use computer algorithms to help patients self-diagnose. The cases were paragraph-long scenarios that included the symptoms and history of the patient, but no physical exam or test findings, and ranged from very serious diagnoses to benign situations, said Dr. Ateev Mehrotra, author of the study and associate professor of health care policy and medicine at Harvard Medical School.
According to the research letter, there were 15 high, 15 medium, and 15 low-acuity condition scenarios and 26 common and 19 uncommon condition scenarios.
The researchers then took those 45 cases and went to the symptom checkers to find out how often they came up with the accurate diagnosis. Those results were published in a separate 2015 study.
Researchers then disseminated those same 45 cases between December 2015 and May 2016 to physicians using the Human Diagnosis Project, or Human Dx, an online platform where doctors can share their own clinical cases or submit answers to others.
Each case was solved by at least 20 physicians, according to the research letter. The diagnoses submitted by the participating physicians were reviewed by two separate physicians who independently decided whether the participants listed the correct diagnosis first or in the top three diagnoses.
The results showed that physicians are far more accurate than the online symptom checkers when it comes to diagnosing cases. The physicians listed the correct diagnosis first across all cases 72.1% of the time, while symptom checkers listed the correct diagnosis first only 34% of the time, according to the research letter.
Physicians also listed the correct diagnosis in their top three diagnoses 84.3% of the time, while symptom checkers included the correct condition in the top three 51.2% of the time.
According to the research letter, physicians were also more likely to list the correct diagnosis first for high-acuity cases compared with low-acuity and for uncommon cases compared with common ones. Symptom checkers, on the other hand, were more likely to list the correct diagnosis first for low-acuity and more common cases.
For example, physicians listed the correct diagnosis first for 79.1% of the high-acuity cases, but 65.3% of low-acuity cases, the researchers found. They listed the correct diagnosis first for 75.5% of uncommon cases, and 69.6% of common cases.
As for the symptom checkers, they listed the correct diagnosis first in 24.3% of high-acuity cases, but 40.5% of low-acuity ones. Symptom checkers offered up the diagnosis first in 28.1% of the uncommon cases, but 38.1% of common cases, according to the research letter.
“I wasn't surprised that the physicians would perform better,” Mehrotra said, adding that while physicians fare better than computers now, “I wouldn't be surprised in the coming decade if we develop computer programs that do make diagnoses as accurate as physicians.”
Mehrotra said it would be useful to study how computers could improve physicians' diagnostic accuracy. Though physicians outperformed the symptom checkers in all scenarios, they provided the incorrect diagnosis in about 15% of cases, the researchers found.
That statistic is in line with other studies, if not higher. But the JAMA study's authors note that there are several limitations. The report uses clinical cases, which may not reflect complex, real-world patients, and the participating physicians also did not have physical exams or test results to help in their diagnosis.
About 12 million U.S. adults experience outpatient diagnostic errors every year, according to a 2014 study published in the BMJ. In that study, researchers estimated the rate of outpatient diagnostic errors at 5%. Half of those errors could be potentially harmful, the study found.
Another 2013 BMJ study by Mark Graber at the SUNY Stony Brook School of Medicine evaluated eight different methods of measuring the rate of diagnostic error among physicians. Some methods, he said, were more accurate than others, and most methods underestimated the actual rate of error.
Among some of the examples outlined in the BMJ report were studies that used standardized patients, or secret shoppers, to measure the rate of diagnostic error. Those methods reported average error rates of 13-15% in diagnosing common conditions. Studies that analyzed autopsy data identified diagnostic errors in 10-20% of cases, Graber found. And studies of error based on second reviews, in which a second physician reviews the same case as the first, have produced estimates of diagnostic error ranging from 10-50%.
There is little data on diagnostic error, and experts say it's hard to track, one reason being that physicians are often reluctant to report mistakes out of fear of litigation.
But there is a hefty price tag that comes with an error in diagnosis.
According to an estimate in a 2015 report from the Institute of Medicine of the National Academies of Sciences, Engineering, and Medicine, the mean payout per diagnostic error in 2011 was $386,849.