With all the recent excitement around using artificial intelligence (AI) to transform healthcare research, delivery, and revenue cycle management, it’s clear there are outstanding opportunities for AI to make a meaningful impact on patient care, research, and operations.
In this conversation, Nasim Eftekhari, executive director of applied AI and data science at City of Hope, one of the largest cancer research and treatment organizations in the U.S., and Erik Pupo, director of commercial health IT advisory at Guidehouse, share best practices and real-world examples for healthcare leaders to strategically apply AI in different aspects of research, care delivery, operations, and business management.
Pupo is a former academic medical center chief information officer with more than 25 years of experience. In his role at Guidehouse, Pupo helps healthcare organizations modernize, engineer, and transform IT infrastructure. At City of Hope, Eftekhari oversees the teams responsible for applying machine learning across the enterprise.
This is an edited transcript of their conversation.
The strategy behind successful Al implementation in healthcare, precision medicine and revenue cycle management
(Left to right) Nasim Eftekhari, executive director of applied AI and data science at City of Hope, and Erik Pupo, director of commercial health IT advisory at Guidehouse
Nasim Eftekhari: City of Hope has an applied AI and data science department, which services all areas of the enterprise with meaningful applications of AI and machine learning. We have three teams within the department. The first team is clinical decision support. We have another team that focuses on precision medicine and research. The final team is the business operations team. This is the team that focuses, in part, on revenue cycle management, finance, and other business applications—anywhere in operations where we can introduce efficiency via use of AI and machine learning. City of Hope has many predictive models we’ve developed in-house over the last seven years to assist real-time clinical decision-making, precision medicine and research operations, in addition to finance applications such as cash and revenue forecasting and revenue cycle optimization. Most recently, we have been experimenting with applying generative AI technology to enhance all of these applications.
Eftekhari: The most important lesson I have learned over the years is that operational support and adoption are more important than the technology itself. What we have found to be the most helpful strategy to increase adoption is to not develop something for the end users but to develop it with them. We have regular meetings with users throughout the entire process—from data collection to development through to testing and deployment, and even after go-live—to make sure we are optimizing the tool to the best of our ability to meet each team’s needs. In a few instances, we still conduct surveys for models that have been in production for years to make sure that the tools continue to meet expectations. User education is another important aspect to ensure that end users are aware of not only the capabilities, but more importantly, the limitations of these AI models and what that means for their workflows. Finally, with any AI model—but more so with generative AI—it’s important to keep humans in the loop. AI tools in healthcare are mostly developed to improve efficiency and enhance workflows, not to replace humans.
Erik Pupo: I would stress data governance and data quality as best practices. Some healthcare organizations trying to do AI and machine learning work haven’t met the minimum level of data governance and data quality you would need to have an impact. Data quality means garbage in, garbage out. If it's not good data you're working with, the results just aren't going to be good. The same thing applies to data governance. If you don't have the right cross-functional teams that work on generative AI services across the entire organization, you're going to have different teams working on a lot of the same data that just aren't aligned and integrated, which leads to wasted resources. And that's another best practice— you need to take a close look in your organization at how you handle the true cost and return analysis of generative AI. Measure these types of efforts, not just on a financial basis, but also on productivity. We’ve seen examples where just the technical cost alone of AI has been a real shocker for CIOs, CFOs and CEOs.
Eftekhari: I couldn’t agree more on measuring the impact. AI has always been—and even more so these days—this shiny new toy that everybody wants to have. Some people may think that AI is something you can take off a shelf and it's going to solve all problems, but that’s not the case. AI is a powerful tool, but it’s still just a tool. We should know how to use AI to solve a problem and make an impact. It’s important to develop metrics for measuring the impact ahead of time and to manage expectations about what the final product or solution may be able to achieve.
Pupo: If you must use patient data, apply the same level of control you would across other types of situations and scenarios where you're working with sensitive data. Oftentimes, the teams that do this work don’t always understand what protected health information is. Also, if aspects of patient consent or privacy are needed, make sure that you're involving your legal and compliance teams.
Eftekhari: It's very important to have overall best practices for data privacy when considering any data product, and AI applications are no exception. When it comes to AI specifically, each application has its own nuances and intricacies and, therefore, should be reviewed by compliance and data privacy teams to make sure the way data is being used is compliant, ethical, safe and responsible.
Eftekhari: The most important aspect of adopting any AI tool is validation and testing. City of Hope tries not to invest in a tool or a partnership before we know it’s going to work for our use cases and patient population. We have seen time and time again that a tool seems to work well for another health system, and doesn't work for us and our patient population. If you are considering co-development with a third party, it’s helpful to look at the type of talent that is assigned to you—that can tell you a lot about the quality of the final solution.
Pupo: You want to look at the vendor's history in terms of data privacy and security before you partner with them. Additionally, with AI in the revenue cycle, it's more about augmented intelligence—you're trying to improve and automate processes. Therefore, when you’re considering a vendor, you should be asking, does this vendor focus more on the process side? Can they come in and provide an immediate ROI through automation?
