Policing the ethics police: Research review boards face scrutiny as feds propose new rules

Institutional review boards (IRBs), designed to serve as gatekeepers protecting patients from unethical human research practices, are under new scrutiny from federal regulators and ethicists.

Critics say studies with serious design flaws that could lead to unethical practices or participant harm continue to win approval because IRBs are too lax, have unqualified members, are riddled with institutional and personal conflicts of interest, and are overwhelmed by the volume of studies they must review.

Those involved with IRBs counter that while the system isn't perfect, the integrity of human research would be compromised without their involvement. They say IRBs have improved in terms of transparency and conflict disclosure.

Last month, the consumer advocacy group Public Citizen and the American Medical Student Association complained to HHS' Office for Human Research Protections (OHRP) about the ethical integrity of two ongoing studies analyzing the impact of longer-than-recommended work shifts for first-year medical residents on 30-day patient mortality rates. Public Citizen and the student association called the studies “highly unethical” for not seeking informed consent from the medical residents or the patients studied. The research design, they said, forced hundreds of residents across the country to work “dangerously long shifts,” placing them and their patients at risk of serious harm. HHS said the OHRP is reviewing the complaints.

The lead investigators on those two studies, from Northwestern University and the University of Pennsylvania, say their studies were evaluated and approved by IRBs at dozens of participating sites. They say there is no solid evidence about whether shorter resident hours are safer for patients, which is why the studies are needed.

“We just have never had prospective, randomized, high-level evidence to inform our decisionmaking,” said one of the study leads, Dr. Karl Bilimoria, vice chairman for quality in the surgery department at Northwestern's Feinberg School of Medicine. “That's why these trials are so important.”

In September, the OHRP and 15 other federal agencies proposed the first substantial revisions in nearly 25 years to the Common Rule, a set of regulations designed to protect medical research participants. The proposed revisions address major shifts in the medical research environment since the early 1990s, and would apply to all U.S. clinical trials, regardless of their funding source. Two provisions would directly affect IRBs.

But Dr. Robert Klitzman, director of the graduate bioethics program at Columbia University, said the proposed rules don't do enough to hold IRBs more accountable and don't address what he calls shortcomings in the qualifications and expertise of IRB members.

Klitzman, who interviewed 46 chairmen, members and administrators from 34 IRBs for his new book, The Ethics Police? The Struggle to Make Human Research Safe, said conflicts of interest on IRBs may arise from personal or collegial relationships. “We know very little about how IRBs make decisions,” he said. “There is no feedback loop, they are not accountable to anyone, and there is no external appeals process or required training to participate.”

In September, researchers from the University of South Florida and the University of Virginia published an article in the European journal Acta Informatica Medica questioning whether members of IRBs fully understand study design. They surveyed 148 individuals with medical or doctoral degrees who participate on IRBs at three dozen U.S. research universities. Examples of survey questions included asking whether a randomized control trial needed a placebo control group; defining the purpose of a Phase I trial; and explaining the purpose of randomization.

Overall, 26.5% of those surveyed answered all of the questions correctly. On the six questions specific to randomized trials, 34% got a perfect score.

The researchers called their findings “alarming,” saying they showed that “those who are charged with oversight in the protection of research integrity may not be adequately equipped to fulfill their duties.”

Such findings could undermine the confidence of researchers and patients who rely on IRBs to provide rigorous safety and ethics reviews of proposed studies. “Too often (IRBs) accept whatever the investigator is telling them and don't ask the right questions,” said Dr. Michael Carome, director of Public Citizen's health research group. “Therefore, they fail to detect the flaws in research design, the ethical problems and the risks.”

There also are worries about “approval fatigue” among IRB members based on the sheer volume of studies they review each year. According to the OHRP, there are 3,547 registered IRBs that review research supported by HHS or that are regulated by the Food and Drug Administration. Those IRBs review more than 675,000 research protocols annually.

There are many more IRBs at academic centers, hospitals, health systems and for-profit research groups that are not required to register with the federal government, but there are no data on exactly how many exist.

The OHRP requires registered IRBs to include at least five members with various specific backgrounds, including at least one scientist, such as a physician or doctoral-level researcher. Also, the board generally includes other healthcare professionals such as nurses. In addition, IRBs must have “unambiguously non-scientific” members, such as ethicists, attorneys or community laymen members.

The federal agency does not systematically monitor or evaluate review boards. But it says members must be “sufficiently qualified” in expertise, diverse in gender and ethnicity, and reflect “community attitudes.” It also says members should not be allowed to review protocols where they have a conflict of interest. The OHRP can act when there are “substantive written allegations or indications of noncompliance with HHS regulations.”

Critics noted a number of controversies over studies that they say raise questions about the standards, transparency and expertise of IRBs. In 2005, the University of Pennsylvania agreed to pay $517,496, and Children's National Medical Center in Washington, D.C., paid $514,622 to the federal government to resolve allegations of false statements and claims associated with a gene therapy study conducted by the two institutions. The family of an Arizona teen who died during the trial said the researchers did not disclose all of the study's anticipated toxicities. That clinical trial continued for a year before it was shut down, even though several patients experienced significant liver toxicities.

Public Citizen previously criticized a National Institutes of Health-funded trial known as the Surfactant, Positive Pressure, and Oxygenation Randomized Trial, which ran from 2005 to 2009. Premature babies were randomized to receive one of two oxygen regimens to see which was less likely to cause blindness. Critics said the participating academic medical centers, including the University of Alabama, Brown, Duke, Stanford and Yale, did not fully inform parents of the trial's risks, such as retinal damage, neurologic injury and death.

The OHRP examined that study's protocols and concluded that its researchers had failed to adequately warn prospective parents about the risks to their newborns posed by participation in the study. NIH Director

Dr. Francis Collins later co-authored a commentary in the New England Journal of Medicine disagreeing with the OHRP's findings.

In another research controversy cited by IRB critics, lawsuits were filed against the Kennedy Krieger Institute over a study from the 1990s on the hazards of pediatric exposure to lead paint. The institute is affiliated with Johns Hopkins University, whose IRB examined and approved the study protocol.

The plaintiff in a case from 2001 claimed that as part of that study, institute researchers tested his home for lead. But researchers did not reveal for nine months that they found high lead levels in that home. The plaintiff also said the study's consent documents did not clearly disclose that participating children might accumulate measurable levels of lead in their blood. The court found that the research protocol was not within an acceptable level of risk because participants who were exposed to high lead levels did not receive treatment. But Kennedy Krieger won dismissal of a similar lawsuit last year in Baltimore.

IRBs often are tasked with monitoring ongoing studies after their initial approval, reviewing changes to research protocols and informed consent documents, and evaluating problems that arise. But they are not always required to do so, and the FDA says its guidance for IRB follow-up review is nonbinding.

Last month's controversy over the Northwestern and University of Pennsylvania resident work-hours studies grew out of concern that patients would be placed at greater risk by residents working longer hours than recommended by the Accreditation Council for Graduate Medical Education.

“Anything that involves putting patients at serious risk, especially by going against recommendations … demands more consent from patients,” said Arthur Caplan, director of the division of medical ethics at New York University's Langone Medical Center.

But the flap confounded other experts. Dr. Mark Friedberg, a senior natural scientist at the RAND Corp. who focuses on medical quality and performance improvement, said he was surprised by the critics' assumption that longer work hours automatically meant increased risks to patients. Shorter work hours mean more patient handoffs between residents, which heightens the risk of miscommunication and medical errors, he said. “It's not clear-cut. That's why you have to do the science.”

Researchers disagree over whether the proposed revisions to the Common Rule affecting IRBs will be positive or negative. One controversial recommendation advocates using a single institutional review board for multisite research studies rather than relying on an IRB at each site.

Supporters of the recommendation say it will reduce burdensome requirements that slow down research. Delays are often caused by “multiple IRBs requesting minor changes to proposed research with limited, if any, gains in the protection of human subjects,” Nicholas Bauroth, vice chairman of the IRB at North Dakota State University, and his colleagues said in written comments on the rule.

But Drs. Ann Arvin and Harry Greenberg, the dean and associate dean for research at Stanford's School of Medicine, wrote that the proposed change advocating a single IRB for multisite studies would mean that an individual institution could not effectively safeguard human subjects enrolled at its site. Furthermore, they said the change would lead researchers to engage in “IRB shopping … for a more-rapid or less-rigorous review.”

The proposed rules are open for public comment through Jan. 6. More than 450 comments had been posted as of Dec. 9.

Elisa Hurley, executive director of the not-for-profit Public Responsibility in Medicine and Research, said her group's main concern is that the Common Rule changes could reduce patient-safety protections by allowing researchers more freedom to independently decide whether their studies need external IRB review. Her group offers a certification program for IRB members.

IRBs, she said, play an important role that shouldn't be downsized. The regulations “overlook the range of ethical issues that an IRB can uncover … even for research that, on its face, looks like it's minimal risk. Investigators themselves may not be in the best position to determine those risks,” Hurley said.