Skip to main content
Sister Publication Links
  • ESG: THE NEW IMPERATIVE
Subscribe
  • Sign Up Free
  • Login
  • Subscribe
  • News
    • Current News
    • COVID-19
    • Providers
    • Insurance
    • Government
    • Finance
    • Technology
    • Safety & Quality
    • Transformation
    • People
    • Regional News
    • Digital Edition (Web Version)
    • Patients
    • Operations
    • Care Delivery
    • Payment
    • Midwest
    • Northeast
    • South
    • West
  • Digital Health
  • Insights
    • ACA 10 Years After
    • Best Practices
    • Special Reports
    • Innovations
  • Opinion
    • Bold Moves
    • Breaking Bias
    • Commentaries
    • Letters
    • Vital Signs Blog
    • From the Editor
  • Events & Awards
    • Awards
    • Conferences
    • Galas
    • Virtual Briefings
    • Webinars
    • Nominate/Eligibility
    • 100 Most Influential People
    • 50 Most Influential Clinical Executives
    • Best Places to Work in Healthcare
    • Excellence in Governance
    • Health Care Hall of Fame
    • Healthcare Marketing Impact Awards
    • Top 25 Emerging Leaders
    • Top 25 Innovators
    • Diversity in Healthcare
      • - Luminaries
      • - Top 25 Diversity Leaders
      • - Leaders to Watch
    • Women in Healthcare
      • - Luminaries
      • - Top 25 Women Leaders
      • - Women to Watch
    • Digital Health Transformation Summit
    • Leadership Symposium
    • Social Determinants of Health Symposium
    • Women Leaders in Healthcare Conference
    • Best Places to Work Awards Gala
    • Health Care Hall of Fame Gala
    • Top 25 Diversity Leaders Gala
    • Top 25 Women Leaders Gala
    • - Hospital of the Future
    • - Value Based Care
    • - Supply Chain
    • - Hospital at Home
    • - Workplace of the Future
    • - Digital Health
    • - Future of Staffing
    • - Hospital of the Future (Fall)
  • Multimedia
    • Podcast - Beyond the Byline
    • Sponsored Podcast - Healthcare Insider
    • Video Series - The Check Up
    • Sponsored Video Series - One on One
  • Data Center
    • Data Center Home
    • Hospital Financials
    • Staffing & Compensation
    • Quality & Safety
    • Mergers & Acquisitions
    • Data Archive
    • Resource Guide: By the Numbers
    • Surveys
    • Data Points
  • MORE +
    • Contact Us
    • Advertise
    • Media Kit
    • Newsletters
    • Jobs
    • People on the Move
    • Reprints & Licensing
MENU
Breadcrumb
  1. Home
  2. Providers
October 10, 2016 01:00 AM

Physicians beat symptom checkers in test of diagnostic accuracy

Shelby Livingston
  • Tweet
  • Share
  • Share
  • Email
  • More
    Reprints Print

    Your doctor may be better than an online symptom checkers to diagnosing what's wrong with you, but he or she still misdiagnoses cases 15% of the time, according to a report published Monday in JAMA Internal Medicine.

    Online symptom checkers are becoming more popular among consumers, so researchers at Harvard Medical School and Boston-based Brigham and Women's Hospital tested whether trained physicians or online symptom checkers were more likely to accurately diagnose a clinical case.

    The researchers used 45 standardized patient cases to test the accuracy of 234 physicians against 23 symptom checkers, which are websites and apps that use computer algorithms to help patients self-diagnose. The cases were paragraph-long scenarios that included the symptoms and history of the patient, but no physical exam or test findings, and ranged from very serious diagnoses to benign situations, said Dr. Ateev Mehrotra, author of the study and associate professor of health care policy and medicine at Harvard Medical School.

    According to the research letter, there were 15 high, 15 medium, and 15 low-acuity condition scenarios and 26 common and 19 uncommon condition scenarios.

    The researchers then took those 45 cases and went to the symptom checkers to find out how often they came up with the accurate diagnosis. Those results were published in a separate 2015 study.

    Researchers then disseminated those same 45 cases between December 2015 and May 2016 to physicians using the Human Diagnosis Project, or Human Dx, an online platform where doctors can share their own clinical cases or submit answers to others.

    Each case was solved by at least 20 physicians, according to the research letter. The diagnoses submitted by the participating physicians were reviewed by two separate physicians who independently decided whether the participants listed the correct diagnosis first or in the top three diagnoses.

    The results showed that physicians are far more accurate than the online symptom checkers when it comes to diagnosing cases. The physicians listed the correct diagnosis first across all cases 72.1% of the time, while symptom checkers listed the correct diagnosis first only 34% of the time, according to the research letter.

    Physicians also listed the correct diagnosis in their top three diagnoses 84.3% of the time, while symptom checkers included the correct condition in the top three 51.2% of the time.

    According to the research letter, physicians were also more likely to list the correct diagnosis first for high-acuity cases compared with low-acuity and for uncommon cases compared with common ones. Symptom checkers, on the other hand, were more likely to list the correct diagnosis first for low-acuity and more common cases.

    For example, physicians listed the correct diagnosis first for 79.1% of the high-acuity cases, but 65.3% of low-acuity cases, the researchers found. They listed the correct diagnosis first for 75.5% of uncommon cases, and 69.6% of common cases.

    As for the symptom checkers, they listed the correct diagnosis first in 24.3% of high-acuity cases, but 40.5% of low-acuity ones. Symptom checkers offered up the diagnosis first in 28.1% of the uncommon cases, but 38.1% of common cases, according to the research letter.

    “I wasn't surprised that the physicians would perform better,” Mehrotra said, adding that while physicians fare better than computers now, “I wouldn't be surprised in the coming decade if we develop computer programs that do make diagnoses as accurate as physicians.”

    Mehrotra said it would be useful to study how computers could improve physicians' diagnostic accuracy. Though physicians outperformed the symptom checkers in all scenarios, they provided the incorrect diagnosis in about 15% of cases, the researchers found.

    That statistic is in line with other studies, if not higher. But the JAMA study's authors note that there are several limitations. The report uses clinical cases, which may not reflect complex, real-world patients, and the participating physicians also did not have physical exams or test results to help in their diagnosis.

    About 12 million U.S. adults experience outpatient diagnostic errors every year, according to a 2014 study published in the BMJ. In that study, researchers estimated the rate of outpatient diagnostic errors at 5%. Half of those errors could be potentially harmful, the study found.

    Another 2013 BMJ study by Mark Graber at the SUNY Stony Brook School of Medicine evaluated eight different methods of measuring the rate of diagnostic error among physicians. Some methods, he said, were more accurate than others, and most methods underestimated the actual rate of error.

    Among some of the examples outlined in the BMJ report were studies that used standardized patients, or secret shoppers, to measure the rate of diagnostic error. Those methods reported average error rates of 13-15% in diagnosing common conditions. Studies that analyzed autopsy data identified diagnostic errors in 10-20% of cases, Graber found. And studies of error based on second reviews, in which a second physician reviews the same case as the first, have produced estimates of diagnostic error ranging from 10-50%.

    There is little data on diagnostic error, and experts say it's hard to track, one reason being that physicians are often reluctant to report mistakes out of fear of litigation.

    But there is a hefty price tag that comes with an error in diagnosis.

    According to an estimate in a 2015 report from the Institute of Medicine of the National Academies of Sciences, Engineering, and Medicine, the mean payout per diagnostic error in 2011 was $386,849.

    Letter
    to the
    Editor

    Send us a letter

    Have an opinion about this story? Click here to submit a Letter to the Editor, and we may publish it in print.

    Recommended for You
    rural-health1.png
    Transgender patients in rural states struggle to find doctors
    The Check Up: John Nickens, LCMC Health
    The Check Up: John Nickens, LCMC Health
    Most Popular
    1
    More healthcare organizations at risk of credit default, Moody's says
    2
    Centene fills out senior executive team with new president, COO
    3
    SCAN, CareOregon plan to merge into the HealthRight Group
    4
    Blue Cross Blue Shield of Michigan unveils big push that lets physicians take on risk, reap rewards
    5
    Bright Health weighs reverse stock split as delisting looms
    Sponsored Content
    Modern Healthcare A.M. Newsletter: Sign up to receive a comprehensive weekday morning newsletter designed for busy healthcare executives who need the latest and most important healthcare news and analysis.
    Get Newsletters

    Sign up for enewsletters and alerts to receive breaking news and in-depth coverage of healthcare events and trends, as they happen, right to your inbox.

    Subscribe Today
    MH Magazine Cover

    MH magazine offers content that sheds light on healthcare leaders’ complex choices and touch points—from strategy, governance, leadership development and finance to operations, clinical care, and marketing.

    Subscribe
    Connect with Us
    • LinkedIn
    • Twitter
    • Facebook
    • RSS

    Our Mission

    Modern Healthcare empowers industry leaders to succeed by providing unbiased reporting of the news, insights, analysis and data.

    Contact Us

    (877) 812-1581

    Email us

     

    Resources
    • Contact Us
    • Advertise with Us
    • Ad Choices Ad Choices
    • Sitemap
    Editorial Dept
    • Submission Guidelines
    • Code of Ethics
    • Awards
    • About Us
    Legal
    • Terms and Conditions
    • Privacy Policy
    • Privacy Request
    Modern Healthcare
    Copyright © 1996-2023. Crain Communications, Inc. All Rights Reserved.
    • News
      • Current News
      • COVID-19
      • Providers
      • Insurance
      • Government
      • Finance
      • Technology
      • Safety & Quality
      • Transformation
        • Patients
        • Operations
        • Care Delivery
        • Payment
      • People
      • Regional News
        • Midwest
        • Northeast
        • South
        • West
      • Digital Edition (Web Version)
    • Digital Health
    • Insights
      • ACA 10 Years After
      • Best Practices
      • Special Reports
      • Innovations
    • Opinion
      • Bold Moves
      • Breaking Bias
      • Commentaries
      • Letters
      • Vital Signs Blog
      • From the Editor
    • Events & Awards
      • Awards
        • Nominate/Eligibility
        • 100 Most Influential People
        • 50 Most Influential Clinical Executives
        • Best Places to Work in Healthcare
        • Excellence in Governance
        • Health Care Hall of Fame
        • Healthcare Marketing Impact Awards
        • Top 25 Emerging Leaders
        • Top 25 Innovators
        • Diversity in Healthcare
          • - Luminaries
          • - Top 25 Diversity Leaders
          • - Leaders to Watch
        • Women in Healthcare
          • - Luminaries
          • - Top 25 Women Leaders
          • - Women to Watch
      • Conferences
        • Digital Health Transformation Summit
        • Leadership Symposium
        • Social Determinants of Health Symposium
        • Women Leaders in Healthcare Conference
      • Galas
        • Best Places to Work Awards Gala
        • Health Care Hall of Fame Gala
        • Top 25 Diversity Leaders Gala
        • Top 25 Women Leaders Gala
      • Virtual Briefings
        • - Hospital of the Future
        • - Value Based Care
        • - Supply Chain
        • - Hospital at Home
        • - Workplace of the Future
        • - Digital Health
        • - Future of Staffing
        • - Hospital of the Future (Fall)
      • Webinars
    • Multimedia
      • Podcast - Beyond the Byline
      • Sponsored Podcast - Healthcare Insider
      • Video Series - The Check Up
      • Sponsored Video Series - One on One
    • Data Center
      • Data Center Home
      • Hospital Financials
      • Staffing & Compensation
      • Quality & Safety
      • Mergers & Acquisitions
      • Data Archive
      • Resource Guide: By the Numbers
      • Surveys
      • Data Points
    • MORE +
      • Contact Us
      • Advertise
      • Media Kit
      • Newsletters
      • Jobs
      • People on the Move
      • Reprints & Licensing