Investing in the unknown
In terms of what’s actually in use today, that’s about it. There are no magical algorithms than can read a patient’s chart and tell doctors with certainty what’s wrong and what the treatment should be. IBM’s Watson hasn’t yet become all it was cracked up to be. The machine is supposed to recommend cancer treatments (among other tasks), but the system itself has had trouble learning from clinical data. Notably, MD Anderson Cancer Center in Houston put a halt to its Watson project last year after spending more than $62 million on it. Still, Watson IBM executives say the machine is in use at 150 organizations.
AI’s limitations can tell us where the industry should be putting its resources and where researchers should focus.
“There’s a lot of hype about the potential of AI for improving inefficiencies, finding new sources of value and unlocking trapped value,” said Brian Kalis, managing director of digital health and innovation for Accenture’s health business. “A big part of this is stepping back and understanding what the business outcomes you’re trying to achieve are.”
Even if a health system can identify the problems it wants to solve, actually putting AI in place is a big deal. “For those who do not establish their own internal development capability the way that Memorial Sloan Kettering has,” said Ari Caroline, chief analytics officer at the New York cancer center, “vendor costs in the AI space can be substantial and would typically be weighed against other major IT expenditures.”
What’s more, that spending can be risky, since it might be going to startups whose technology has not yet been proven. “Health systems are worried about their current business model and how long it will last,” said Dr. Bob Kocher, a partner at Venrock. “They have very low margins, so they can’t make a lot of speculative R&D developments. That makes them need AI solutions that have a nearly instantaneous payback and are very short-term focused and work very well. They want proof that these things are going to work and going to pay back.”
It’s a tough assignment for a technology that, relatively speaking, isn’t widely adopted, especially in an industry that’s notoriously slow to use new technologies, and especially in an industry that’s just gone through a major—and expensive—upheaval with the implementation of massive EHR systems. “Now that we’ve implemented the EHR, healthcare is trying to determine how to go after the next frontier,” Meadows said. “There’s definitely a concern about costs, because it’s in its infancy. Everything costs more when it’s not a fully vetted product or process. These early adopters will be investing a lot of money to potentially fail.”
Harnessing the data
One point of entry into AI could be EHRs themselves, where the clinical data that algorithms depend on reside. But, as Beth Israel’s Tandon points out, AI isn’t what most EHR vendors specialize in. “Most hospitals depend on their EHR vendors to make innovations,” he said, and “most healthcare organizations don’t have any control over the vendor platform they use.” When the data needed for AI are trapped in EHRs, and the EHR vendors aren’t yet focusing on AI, healthcare organizations are stuck. “The data are in a place where the know-how doesn’t exist, and the know-how is in a place where the data doesn’t exist,” Tandon said.
Whether AI succeeds depends in large part on how available the necessary data are, wrote authors from Jason, a scientific advisory group, in a December 2017 report for HHS about AI in healthcare. “AI application development requires training data and will perform poorly when significant data streams are absent,” they wrote.
While healthcare is awash in data, those data are often not consistent, clean or in sets large enough to “teach” AI algorithms enough to be trustworthy. Just as the lack of interoperability hinders continuity of care and burdens providers, so too does it hinder and burden AI.
“If you’re going to let a machine make a decision for you, you better be darn sure that the data you’re feeding it are good,” Meadows said. “I think healthcare is kind of in a transition, because we’ve worked for years and years to get EHRs in place, and really, those are just transactional systems,” she said. “How do we begin to bring all the data together to make educated decisions and have the cleanliness of data?”
Some healthcare systems are taking the first steps to make sure data are clean—that is, reliable, accurate and free of inconsistencies—from the get-go. “We’ve decided to make our clinical data much more amenable to machine learning and making sure we’re consistently extracting structured clinical features from unstructured text,” Sloan Kettering’s Caroline said, something that’s done either manually or through natural language processing. EHRs can make that difficult, he said, since often clinical information exists only in free text in notes because EHRs were designed to be financial, not clinical, systems.
Getting information out of the EHRs—and keeping it secure in the process—is one thing. There’s also the problem of getting AI insights back into workflows. “How do we insert that back into a workflow to do something, to drive value?” Cleveland Clinic’s Morris said. “If you don’t, you’re just adding cost; you’re just adding tools.”
