Electronic health record company Epic launched an open-source tool on Wednesday that will allow healthcare providers to test and validate artificial intelligence models.
The tool is free and available through the public coding website Github where health systems can add it to their electronic health record systems, said Corey Miller, vice president of research and development at Epic. The tool will allow health systems to evaluate the performance of any AI model that integrates with any EHR, whether it's from Epic or not, Miller said.
Read more: AI challenges: a hot-button topic at Digital Health Summit
“This tool is decoupled from Epic software. So, it in no way gets an Oracle customer closer to Epic,” Miller said. “This just seemed like a real opportunity to meet a need broadly in the industry that wasn't there."
The roll out of this validation tool comes at a time when industry stakeholders are seeking to harmonize standards and reporting for health AI. Despite the hype surrounding AI, health systems lack a standard way to evaluate a model's efficacy.
Epic is calling the tool the AI Trust and Assurance Suite. Using components from another open-source AI tool, the suite can audit machine learning models for discrimination and bias, Miller said. The tool was designed to be flexible and evolve as standards, techniques and approaches develop.
“It goes beyond just model equity, fairness [and] performance, though it does those things too,” Miller said. “But it also looks at what's downstream of the model.”
Epic said the suite can be used to test models developed by third-party vendors or health systems. Providers will be able to test a model's performance against a system's own internal data. A spokesperson for the Madison, Wisconsin-based University of Wisconsin Health said it plans to use the AI tool for model validation but did not specify specific use cases.
The Health AI Partnership, an industry stakeholder group started at Durham, North Carolina-based Duke Health, will support the use of Epic’s tool for its health system members for local AI model validation, a spokesperson said.
It's been a busy week for Epic, which said Tuesday it was working with identity verification company Clear on patient check-in kiosks for health systems.
