Skip to main content
Sister Publication Links
  • ESG: THE NEW IMPERATIVE
Subscribe
  • My Account
  • Login
  • Subscribe
  • News
    • Current News
    • COVID-19
    • Providers
    • Insurance
    • Government
    • Finance
    • Technology
    • Safety & Quality
    • Transformation
    • People
    • Regional News
    • Digital Edition (Web Version)
    • Patients
    • Operations
    • Care Delivery
    • Payment
    • Midwest
    • Northeast
    • South
    • West
  • Digital Health
  • Insights
    • ACA 10 Years After
    • Best Practices
    • Special Reports
    • Innovations
  • Data/Lists
    • Rankings/Lists
    • Interactive Databases
    • Data Points
  • Op-Ed
    • Bold Moves
    • Breaking Bias
    • Commentaries
    • Letters
    • Vital Signs Blog
    • From the Editor
  • Awards
    • Nominate/Eligibility
    • 100 Most Influential People
    • 50 Most Influential Clinical Executives
    • Best Places to Work in Healthcare
    • Excellence in Governance
    • Health Care Hall of Fame
    • Healthcare Marketing Impact Awards
    • Top 25 Emerging Leaders
    • Top 25 Innovators
    • Diversity in Healthcare
    • Women in Healthcare
    • - Luminaries
    • - Top 25 Diversity Leaders
    • - Leaders to Watch
    • - Luminaries
    • - Top 25 Women Leaders
    • - Women to Watch
  • Events
    • Conferences
    • Galas
    • Virtual Briefings
    • Webinars
    • Transformation Summit
    • Women Leaders in Healthcare Conference
    • Social Determinants of Health Symposium
    • Leadership Symposium
    • Health Care Hall of Fame Gala
    • Top 25 Women Leaders Gala
    • Best Places to Work Awards Gala
    • Top 25 Diversity Leaders Gala
    • - Hospital of the Future
    • - Value Based Care
    • - Supply Chain Revenue Cycle
    • - Hospital at Home
    • - Workplace of the Future
    • - Strategic Marketing
    • - Virtual Health
  • Listen
    • Podcast - Next Up
    • Podcast - Beyond the Byline
    • Sponsored Podcast - Healthcare Insider
    • Video Series - The Check Up
    • Sponsored Video Series - One on One
  • MORE +
    • Advertise
    • Media Kit
    • Newsletters
    • Jobs
    • People on the Move
    • Reprints & Licensing
MENU
Breadcrumb
  1. Home
  2. Technology
October 28, 2015 01:00 AM

Few EHRs score well on new AMA usability ratings

Joseph Conn
  • Tweet
  • Share
  • Share
  • Email
  • More
    Reprints Print

    Only three electronic health-record systems got perfect scores for usability—two made by Allscripts and one by McKesson Corp.—out of 20 products rated by the American Medical Association and a group of researchers.

    The AMA teamed up with MedStar Health's National Center for Human Factors in Healthcare to rate 20 commonly used EHRs by their adherence to user-centered design best practices.

    The average score on the 15-point scale was 11.5 and the median was 12.5. The worst-performing product, from eClinical Works, got a 5.

    Vendors were supposed to incorporate user-centered design principles in developing systems that would be eligible for the federal EHR incentive payment system.

    This year researchers at Washington-based MedStar reported that many EHR vendors failed to meet the program's user-centered design requirements while others ignored optional user-centered design best practices.

    “Physician experiences documented by the AMA demonstrate that most EHR systems fail to support effective and efficient clinical work, and continued issues with usability are a key factor driving low satisfaction with many EHR products,” AMA President Dr. Steven Stack said in a news release about the ratings, which are available online free of charge. “Our goal is to shine light on the low bar of the certification process and how EHRs are designed and user-tested in order to drive improvements that respond to the urgent physician need for better designed EHR systems.”

    To qualify their EHR systems for use by providers in the federal incentive payment program, vendors were asked to report to the Office of the National Coordinator for Health Information Technology at HHS how they met their user-centered design requirements. Those requirements included multiple users actually testing the systems for usability and scoring the results.

    These reports are made available to the public via an ONC website, but they are not uniform in design and the placement of key information. Further, their data was not scored against recognized best practices in user-centered design principles, nor was it compiled in one easily accessible format for comparison with usability reports for other vendors' systems.

    In their initial study, the MedStar researchers looked at the ONC filings on 50 EHRs. They found that nine in the sample (18%) had no public report on usability. Of the 41 vendors with reports on file, about a third failed to state the type of user-centered design process they used, a requirement for certification.

    The ONC also has endorsed a National Institute of Standards and Technology recommendation that vendors use at least 15 participants when performing user-centered design testing. But the ONC didn't make that threshold mandatory, and 63% of the vendors in the MedStar sample had fewer than 15 participants in end-user testing.

    The new rating scheme ranks each vendor's EHR on a 15-point scale by comparing what the EHR vendors reported to the ONC against best-practices criteria developed by participants from the AMA and the MedStar center.

    “Every single vendor should be at 15 points,” said Raj Ratwani, scientific director of the MedStar's Human Factors Center and a principal developer of the ratings system.

    For the initial 20 vendors, Ratwani said, he and fellow researcher Dr. A Zach Hettinger, the center's medical director, “went through every single use case these vendors used and how they approached user-centered design.” Dr. Michael Hodgkins, vice president and chief medical information officer for the AMA, and Matt Reid, the association's senior health IT consultant, also worked on the ratings framework.

    The ONC did not respond to requests for comment on the new rating system. More than 400 EHR systems have been tested and certified for use in the federal incentive payment program, so Ratwani acknowledged the ratings are a work in progress.

    “If we can spark a little bit of competition with this framework, then we're achieving our goal of improving these products,” he said.

    The Electronic Health Records Association, a trade group representing 31 EHR vendors, panned the ratings as mostly useless to buyers.

    “We note the acknowledgement in the AMA's press release that the framework is not intended to evaluate the perceived usability of an EHR, only the documentation submitted for certification,” the association said in an e-mail. “Given that the documentation presented for certification is a small subset of the usability evaluations and user-centered design techniques that a vendor may practice, this framework may not represent a comprehensive view of the user-centered design work done in the context of any given product.”

    Letter
    to the
    Editor

    Send us a letter

    Have an opinion about this story? Click here to submit a Letter to the Editor, and we may publish it in print.

    Recommended for You
    Cue Health COVID test
    High-tech’s business model hasn’t worked for the Cue COVID test
    freestocks-ux53SGpRAHU-unsplash_7.jpg
    With Roe in doubt, some fear tech surveillance of pregnancy
    Sponsored Content
    Health IT Strategist (HITS) Newsletter: Sign up for the latest IT and medical technology news delivered 3 days a week (M, W, F).
     
    Get Newsletters

    Sign up for enewsletters and alerts to receive breaking news and in-depth coverage of healthcare events and trends, as they happen, right to your inbox.

    Subscribe Today
    MH Magazine Cover

    MH magazine offers content that sheds light on healthcare leaders’ complex choices and touch points—from strategy, governance, leadership development and finance to operations, clinical care, and marketing.

    Subscribe
    Connect with Us
    • LinkedIn
    • Twitter
    • Facebook
    • RSS

    Our Mission

    Modern Healthcare empowers industry leaders to succeed by providing unbiased reporting of the news, insights, analysis and data.

    Contact Us

    (877) 812-1581

    Email us

     

    Resources
    • Contact Us
    • Advertise with Us
    • Ad Choices Ad Choices
    • Sitemap
    Editorial Dept
    • Submission Guidelines
    • Code of Ethics
    • Awards
    • About Us
    Legal
    • Terms and Conditions
    • Privacy Policy
    • Privacy Request
    Modern Healthcare
    Copyright © 1996-2022. Crain Communications, Inc. All Rights Reserved.
    • News
      • Current News
      • COVID-19
      • Providers
      • Insurance
      • Government
      • Finance
      • Technology
      • Safety & Quality
      • Transformation
        • Patients
        • Operations
        • Care Delivery
        • Payment
      • People
      • Regional News
        • Midwest
        • Northeast
        • South
        • West
      • Digital Edition (Web Version)
    • Digital Health
    • Insights
      • ACA 10 Years After
      • Best Practices
      • Special Reports
      • Innovations
    • Data/Lists
      • Rankings/Lists
      • Interactive Databases
      • Data Points
    • Op-Ed
      • Bold Moves
      • Breaking Bias
      • Commentaries
      • Letters
      • Vital Signs Blog
      • From the Editor
    • Awards
      • Nominate/Eligibility
      • 100 Most Influential People
      • 50 Most Influential Clinical Executives
      • Best Places to Work in Healthcare
      • Excellence in Governance
      • Health Care Hall of Fame
      • Healthcare Marketing Impact Awards
      • Top 25 Emerging Leaders
      • Top 25 Innovators
      • Diversity in Healthcare
        • - Luminaries
        • - Top 25 Diversity Leaders
        • - Leaders to Watch
      • Women in Healthcare
        • - Luminaries
        • - Top 25 Women Leaders
        • - Women to Watch
    • Events
      • Conferences
        • Transformation Summit
        • Women Leaders in Healthcare Conference
        • Social Determinants of Health Symposium
        • Leadership Symposium
      • Galas
        • Health Care Hall of Fame Gala
        • Top 25 Women Leaders Gala
        • Best Places to Work Awards Gala
        • Top 25 Diversity Leaders Gala
      • Virtual Briefings
        • - Hospital of the Future
        • - Value Based Care
        • - Supply Chain Revenue Cycle
        • - Hospital at Home
        • - Workplace of the Future
        • - Strategic Marketing
        • - Virtual Health
      • Webinars
    • Listen
      • Podcast - Next Up
      • Podcast - Beyond the Byline
      • Sponsored Podcast - Healthcare Insider
      • Video Series - The Check Up
      • Sponsored Video Series - One on One
    • MORE +
      • Advertise
      • Media Kit
      • Newsletters
      • Jobs
      • People on the Move
      • Reprints & Licensing