Digital tools give providers a lot of data, but can clinicians trust it?

Investment in digital medicine has exploded in the past few years, with mobile apps and wearables unfurling vast amounts of previously unknown data to consumers, providers and health plans. But experts worry that a lack of standardized measures could slow provider adoption of digital tools.

“I can’t imagine purchasing a piece of biomedical equipment without proof of validation,” said Dr. Edmondo Robinson, senior vice president and chief digital officer at Moffitt Cancer Center in Tampa, Florida. “I would never unleash that on a patient, right? It would be wrong. So why would I do that with digital tools?”

Clinically validated digital endpoints—sensor-generated data collected from patients at home or other non-clinical settings—could help manage conditions that are costly and pervasive. However, few digital tools on the market have been formally approved like medical devices. While clinical labs and device makers must follow strict guidelines to ensure that test results and measurements are comparable, there are no similar standards for digital endpoints.

“The absence of clinical validation reduces the motivation for the parties to use these tools,” said Dr. Kaveh Safavi, a consultant at Accenture.

Moreover, few vendors have strong evidence that their solutions deliver the results they promise because their investors don’t require it, leaving it up to healthcare executives to separate fact from fiction.

“The big venture capital firms will give you hundreds of millions of dollars with no evidence, so long as you show some kind of engagement or you’re selling your product,” Robinson said.

Confronting the challenges

Validating digital endpoints could prove challenging, especially for conditions without an established body of knowledge or standard of care for judging clinical validity, Safavi said. For example, it could be difficult for researchers to validate behavioral health-related endpoints because there are only a few outcome indicators for care quality.

“We’ve got all these very useful self-help tools that have incorporated various methodologies like cognitive behavioral therapy,” Safavi said. “But at the end of the day, you have to ask the question: ‘Is this working?’ ”

Safavi was involved in an effort by the American Medical Association to establish standards for evaluating digital tools. But it didn’t go anywhere because there was no organization to do clinical validation, he said.

“Everything we did basically was testing whether or not the technology itself met privacy and security standards. But it didn’t solve the clinical validation problem. The clinicians’ perspective was, ‘That’s all great, but I still don’t know if this stuff works.’ ”

Digital health startups are beginning to take notice, responding by funding and publishing studies that demonstrate their solutions are effective. For instance, a 2021 study published in the Journal of Medical Internet Research found that 59% of Ginger patients reported reduced anxiety after eight to 12 weeks using a combination of the mental health company’s text-based coaching, therapy and psychiatry services.

Validating quality, cost and efficiency outcomes can help vendors stand out among their competition, said Rosemary Kennedy, chief health informatics officer for Connect America, a company that uses technology to help at-risk people live independently.

“A focused attention on clinician and patient usability is equally important,” Kennedy said.

But many studies are small and haven’t been tested in the real world. Without support from leading medical experts, specialty societies, or inclusion in care standards, digital tools and the sensor-generated data they rely on won’t clear the bar of clinical validation for many providers.

Download Modern Healthcare’s app to stay informed when industry news breaks.

The Digital Medicine Society (DiMe)has tried to address the issue by creating a library of digital endpoints. But the effort has largely illustrated the scope of the problem rather than offering a solution. The federal government hasn’t approved a new medical product using sensor data in the past two years, even though there has been a 665% increase in unique sensor-generated health data in industry-sponsored clinical trials, according to DiMe.

New therapies languish in the Food and Drug Administration’s approval process because regulators need to figure out whether the sensor data is valid before they evaluate the intervention itself.

DiMe is leading an effort to establish nocturnal scratch as an evidence-based digital measure for eczema with support from pharmaceutical companies, including AbbVie, Janssen Research & Development, Novartis, Pfizer and UCB. The companies had been working separately on similar projects, said Jennifer Goldsack, DiMe’s CEO. The aim is to create a proven measure that drugmakers, medical device companies, researchers and others can use to develop new ways to diagnose and treat people with eczema. DiME hopes the initiative will create a blueprint for developing digital endpoints in medical products.

Doing it on their own

In the meantime, some large providers are investing in pilots and partnerships to establish their own approaches to clinically validating digital solutions by focusing on their most common, highest-cost conditions.

Louisiana’s Ochsner Health launched its Connected Maternity Online Monitoring program in 2017. The program uses digitally connected scales to monitor body weight of pregnant women. Ochsner trains patients to take their blood pressure readings from home; the data is sent directly from a patient’s smartphone to the health system’s electronic health record system using Epic’s MyChart app.

Ochsner published research showing that its Connected MOM program can identify masked hypertension earlier by capturing vitals at home, leading to earlier diagnosis and lower rates of pre-term deliveries than traditional obstetric care. Not only has the program improved patient outcomes, but it’s also reduced the average number of office visits by three, said Dr. Richard Milani, Ochsner’s chief clinical transformation officer and vice chairman of cardiology. Ochsner has similar programs for diabetes, hypertension, fall prevention and other conditions.

Unfortunately, most hospitals and health systems can’t devote the resources necessary to build or validate solutions in-house, needing to rely on solutions validated by others.

Crucial to healthcare transformation

Without a coordinated, systematic approach to digital medicine, healthcare executives risk overwhelming their technology and staff.

While several vendors offer solutions to address specific conditions and care challenges, trying to integrate too many of them can strain a health system’s data infrastructure. Vendor-specific sensor data can cause a host of issues if they need unique interfaces, data containers or other one-off technical requirements.

“One of the most important things, especially when it comes to clinical digital endpoints, is to avoid creating a complex and confusing mosaic of 500 individual point solutions that don’t speak to one another,” Goldsack said.

Clinically validated digital endpoints could help providers re-imagine their infrastructure and clinical workflows by allowing them to use a common set of proven data across therapeutic areas, reducing complexity and enabling providers to focus on what matters most. Clinical validity could also help curb the impact of labor shortages since wearables allow for remote and consistent monitoring.

Payers might also start to require validation to achieve quality goals for value-based payment, said Joe Lynch, a validation expert at Avalere Health.

Third-party accreditation for digital health tools might be a viable pathway for establishing a minimal level of clinical validity without the need to go through an established FDA pathway, according to experts. But no such accreditation exists.

“The bar will be lower, but it won’t be zero,” Safavi said.

New interoperability standards like the Fast Healthcare Interoperability Resources standard and open application programming interfaces could speed adoption of validated endpoints by accelerating the separation of data and applications in healthcare. “You want data that are generated in one place 
to be available for use in another place,” Safavi said.

Though clinical validation won’t guarantee the success of digital medicine in the long run, it’s hard for experts to see how digital tools will reach their potential if clinicians don’t understand and trust the data they provide. “If a digital endpoint is fully, clinically validated, you know what it means. You know what that data point is telling you,” Goldsack said. “That is a gift.”