As health systems and insurance companies ramp up adoption of artificial intelligence and machine learning technology, experts fear clinical algorithms are not ready for prime time—with potential consequences for patient safety and outcomes.
The White House Office of Science and Technology Policy released a blueprint for an “AI Bill of Rights” in October to help healthcare and other sectors navigate the potential perils of the technology. But the development of AI in healthcare has greatly outpaced fledgling government efforts to control it. Some leading health systems are putting their own guardrails in place. Finding the tech talent needed to oversee an expansive, self-regulated AI division has proven challenging for others, however.
While 85% of healthcare leaders said in a Deloitte survey from June that they expect to increase their AI investments in the next year, only 57% say their organization is prepared to handle failure or bad decisions stemming from use of AI. Nearly half are not prepared to handle new or changing regulations concerning AI.
“It’s the Wild West and it’s becoming embedded in every organization,” said Keith Figlioli, partner at venture firm LRVHealth. Before joining LRV, Figlioli was a member of the Office of the National Coordinator for Health Information Technology’s Standards Committee for three years.
He said guidelines should be enacted around how an AI model is introduced, tested and applied to various demographics, and health systems should publicly clarify what protocols are in place when something goes wrong.
“The promise of AI is so great that we sometimes forget these algorithms are programmed by people. And assumptions are made by people,” said Dr. Vindell Washington, chief clinical officer for Alphabet’s life sciences firm, Verily, and a former national coordinator for health IT. Washington noted such assumptions could lead to faulty information being fed to models.
“Even if the algorithms are working perfectly, the places and sources of data are sometimes imperfect as they’re collected and delivered,” he said.