Ambient scribes need more research to prove ROI: Peterson Health

The verdict is still out on one of the most popular early use cases of artificial intelligence in healthcare.

A report published Tuesday from digital health research group Peterson Health Technology Institute found many claims by ambient AI vendors need additional research. Peterson's researchers, who spoke to around 60 providers, industry experts and vendors for the report, said while there are benefits of the technology in reducing physicians' cognitive load, some vendor claims may be overstated.

Related: Hospitals establishing hubs for digital health operations

There needs to more defined metrics on how to evaluate these products, said Prabhjot Singh, a senior advisor at Peterson Health Technology Institute. 

“There is just not a consensus on some of the measures needed to determine whether or not [ambient documentation tools] do or don't work,” Singh said. "If I was going to measure this across the enterprise as a chief operating officer or chief analytics officer, would I really know what my core measures are, and would I know how to kind of value them?" 

Hospitals and health systems are increasingly turning to ambient AI documentation tools, which can turn a recording of a doctor-patient conversation into usable clinical notes. As a result, the number of ambient clinical documentation vendors continues to rise. Peterson researchers found there approximately 60 companies were selling products ranging from established technology companies like Microsoft and Oracle, to emerging companies like Abridge, Nabla and Suki.

Here are five additional takeaways from Peterson's report.

1. Pajama time claims from vendors could be overstated

AI vendors often say their tools can lead to a reduction in the amount of time clinicians spend documenting their patient encounters after hours, known as pajama time. But some providers interviewed by Peterson found the tools yielded only modest or no improvement in reducing pajama time. Authors said more research was needed to confirm whether ambient clinical documentation tools reduced the amount of time providers spent documenting in the EHR outside of their scheduled clinic time.

2. The tools appear to have some benefits

Based on the experiences of early adopters, authors found ambient scribes appear to reduce physicians' cognitive load and levels of burnout while improving the patient experience. Both data and anecdotal feedback from respondents found the tools had a positive impact on a clinician's cognitive load. The results back up other anecdotal experiences including Denver Health, which saw an uptick in physicians' happiness after implementing ambient AI tools.

3. Financial return on investment unclear

It is also too early to say if the ambient AI tools will yield a financial return on investment, researchers said. Most organizations reported no increase in the number of patient encounters per period after implementing ambient AI. Additional research is required in this area, the authors wrote. Researchers also found it was too early to understand the effect of ambient scribes on the accuracy or completeness of medical coding for billing purposes.  

4. The space could soon be commoditized

Peterson’s report said ambient clinical scribes are becoming commoditized. The costs to switch between vendors offering competition product is low, researchers wrote. Most of the experts Peterson spoke to for the report expect ambient scribes will have largely penetrated the ambulatory market within the next 24–36 months, which could mean diminishing returns for vendors. 

Researchers also emphasized the diminishing returns from next-generation products. The report said many differences between products have diminished, which has led companies to begin bundling scribe capabilities into broader technology offerings to stand out.

5. Patients seem to be embracing the tools

While many areas of ambient AI evaluation are too nascent to evaluate, researchers are more confident in the effect that ambient scribes has had on patients. Early adopters said the tools are having a positive impact on patient experience in early surveys,