“Early detection” has long been the buzzword in cancer care. So when computer-assisted detection, or CAD, software hit the marketplace 12 years ago, manufacturers and providers alike envisioned a day when the technology would become so sophisticated it might all but eliminate the possibility of missing an early cancer.
Today CAD, which uses computer-generated markings to draw a radiologist's attention to questionable areas on an image, is far from being flawless in its function or ubiquitous in its use. But a growing number of provider organizations are making the technology part of their image-reading options and protocol.
According to a February 2005 study published in the American Journal of Roentgenology, 10% of mammography facilities in the U.S. had adopted CAD technology within three years of the U.S. Food and Drug Administration's approval of the first CAD device in 2002.
Listen to a podcast interview with the ECRI Institute's Jason Launders discussing the pros and cons of CAD“A lot of the CAD systems are acting as an assistant to radiologists—especially in breast imaging—so that we can accurately assess the image,” say Gillian Newstead, a professor of radiology at the 548-bed University of Chicago Medical Center.
CAD currently is being used in breast mammography, lung imaging and colon-cancer screening, but despite its promise, many proponents of the technology say its development has stalled in recent years, and available CAD systems haven't kept pace with the imaging modalities with which they are paired.
“I did an evaluation of mammography CAD systems in 2002, and since then the technology's underlying algorithms haven't changed at all,” says Jason Launders, medical physicist and senior project officer with the healthcare-technology analysis group ECRI Institute based in Plymouth Meeting, Pa.
Other industry experts express similar concern about the state of CAD technology. “There have been no new CAD device applications cleared by the FDA since 2006,” says David Fisher, managing director of the Medical Imaging & Technology Alliance, a group representing radiology equipment manufacturers. “We estimate that between 10 and 20 CAD applications have gone to the FDA, but none have been cleared.”
Fisher declined to say which companies have submitted CAD products for FDA review within the past four years, but he says he believes that manufacturers' and FDA reviewers' confusion over the premarket approval process is to blame for the lack of product approvals.
Now some observers are worried that the FDA's proposed new guidance on the CAD technology approval process could make getting new and updated products into the hands of providers nearly impossible. They say added requirements for clinical studies and design verification will make CAD development too costly, particularly since the technology provides limited reimbursement opportunities for providers. Mammography CAD is the most widely covered form of the device's use, with Medicare and most private payers reimbursing providers for CAD services. Lung CT and colon-screening CAD is less often covered.
“These can be expensive add-on software systems to buy,” Launders says. “It can cost $50,000, so it's the kind of thing hospitals knock off the order when they're looking to buy new imaging technology.”
In October 2009, after two years of work by a radiological device advisory panel, the FDA released proposed guidance for its CAD technology approval process. Contained in two documents, the draft guidance is the agency's first attempt to standardize and clarify the CAD product-approval application process.
“The goal is to let industry know what we hope to see in an application and what criteria we need them to meet,” says Nicholas Petrick, deputy director of the FDA's Center for Devices and Radiological Health's science and engineering laboratories office.
The first of the two documents details proposed new rules for the 510(k) submission process—which covers CAD product updates, moderate-risk devices and new products that are equivalent to already approved CAD technology. The second document directs manufacturers on how to design and conduct their clinical-performance assessment for both 510(k) and new, high-risk CAD products approved under the pre-market application, or PMA, process.
“We think the process they've put forward is excessively burdensome,” Fisher says of the draft guidance.
Under FDA regulations, companies are required to provide more performance evidence and clinical-testing data for PMA applications than for those approved through the 510(k) process. But critics say that the bar for proving the safety and efficacy of CAD products under the 510(k) process would be raised too high if the agency's current draft guidance is approved.
“This substantially bumps the requirements of a 510(k) device up to that of a PMA,” says Robert Nishikawa, a University of Chicago medical physicist who has consulted with a number of companies on radiology product development. The change would likely add time and expense to the approval process.
Under the proposed 510(k) application guidelines, manufacturers would be required to detail how they designed the algorithms that guide their CAD technology to identify potential areas of concern on an image. CAD devicemakers are worried, however, that the requirement would force them to disclose proprietary information.
“We spend millions and millions of dollars to design those algorithms, and we don't want to reveal those trade secrets,” says Julian Marshall, director of product management for workflow solutions for Hologic, a Bedford, Mass.-based developer of mammography CAD software. He argues that the FDA should be able to glean safety and efficacy information from stand-alone testing of CAD devices. But the validity of stand-alone testing is also being challenged by the advisory panel that wrote the guidelines.
Previously, CAD manufacturers have been able to submit stand-alone test analysis—which looks at how accurately a CAD system marks images from previously determined cases—with their 510(k) applications to demonstrate that their devices perform on par with others already on the market. But the FDA's proposed guidance would also require devicemakers to perform clinical studies on 510(k)-eligible devices. The guidance would, in most cases, require devicemakers to conduct multiple reader, multiple case studies where radiologists' readings of an image would be compared against the new CAD system's assessment of an image.
“One of the problems is that the manufacturers don't have access to their competitors' algorithms, so it's difficult for them to know if their product is equivalent,” Petrick says. “So in those cases, I think we'll definitely need to have reader studies that show the product is equivalent.”
Critics of the draft guidance say the requirement is not realistic. “Having run multiple reader, multiple case trials, I'm very sensitive to how difficult logistically it is to put together these studies,” says Abe Dachman, professor of radiology with the University of Chicago Medical Center. “Finding radiologists who are willing to read the cases two and three times is difficult, and companies want to avoid doing multiple reader, multiple case studies because it's so expensive.”
Dachman was unable to estimate the cost of multiple reader clinical studies, but Hologic's Marshall says the expense easily runs into hundreds of thousands of dollars.
The FDA also would require that 510(k) clinical studies include a control group to show if there is a difference between how accurately radiologists assess images when they are aided by CAD technology on a first read and when they are unaided by the technology on a first read.
Certainly, the findings on the benefits of CAD are mixed. An April 5, 2007 New England Journal of Medicine study, Influence of Computer-Aided Detection on Performance of Screening Mammography, found for example that the use of CAD was associated with reduced image interpretation accuracy by physicians and increased rates of biopsy, but was not clearly associated with improved detection of invasive breast cancer.
Another study, Single Reading With Computer-Aided Detection for Screening Mammography, published Oct. 16, 2008, in the same medical journal, found that double reading of a mammogram image by two radiologists produced more accurate conclusions than a single read of an image coupled with CAD assistance.
Newstead says the challenge of CAD technology is that because it is more sensitive to potential anomalies than the human eye, it often calls attention to perfectly healthy tissue. Some studies have found, for example, that CAD typically generates two to four false-positive marks on a single mammography study. That means more patients are likely to be recalled for unnecessary biopsies and other tests, particularly when CAD is being used by a less-experienced radiologist who may not be skilled in determining if a CAD marking is superfluous.
“CAD is not perfect,” says Chris Flowers, an associate professor of radiology with the University of California at San Francisco's breast-imaging program. “It can mark things that aren't an area of concern, and it can miss things that are and that can be dangerous. It can be a distraction to the reader when there's a positive finding that actually represents nothing.”
Flowers believes that the FDA's request for information about algorithm designs and additional clinical studies for CAD 510(k) applications are positive moves given uncertainty about the benefits of CAD. “I'm concerned that anytime you are going to allow a computer-aided device to be involved in detection you have to be careful to make sure the algorithms are accurate.”
No date has been set for the FDA to release its final version of the CAD guidance, Petrick says, but many stakeholders say they hope the final version will do a better job of distinguishing between 510(k) application requirements and the more strenuous requirements for PMA applications, which cover new high-risk devices.
“The definitions of CAD need to be clearer in terms of where individual devices fall in the approval process,” Fisher says. “There are different types of CAD, but right now, it's all a blurry line.”
Petrick acknowledges that while the advisory panel is working “to be as specific as we can” in the final guidance, CAD represents a unique challenge for the agency to regulate as the device is essentially software. “This is important guidance, and we recognize this, so we're working to get it right.”
Some observers believe that the FDA's efforts to require more rigorous clinical proof of CAD devices' safety and efficacy are also a function of the agency's desire to sort out under which usage conditions CAD is most effective in helping clinicians detect disease while simultaneously minimizing the chance of unnecessary testing and procedures being performed on patients.
“We don't know the answer to how doctors interact with CAD, but we do need more studies” of that, Flowers says. He adds, however, that it is unclear if the FDA should be playing a role in providing those answers.
Dachman is less equivocal. “What the FDA wants to be able to do is say, ‘This is how the CAD performed when it was used on a secondary or primary view' ” of the image, he says. “That's an admirable goal, but it's not the real world. What we can say is, ‘Here is how CAD performed in stand-alone trials, and it's up to you to be a responsible provider when using the technology.' ”
