Study Faults Selective Publication of Antidepressant Trials

Medical journals have a unique image in the US health care system. Because most of them adhere to a strict system of critical peer review, they are often seen as unimpeachable sources of accurate information about the safety and efficacy of new medications. But a new study calls into question the completeness of the data offered to-and by-journals.

In "Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy," which appeared in the January 17, 2008, issue of the New England Journal of Medicine, researchers argue that a paucity of unfavorable articles in the medical literature may result in a distorted picture of a drug's effectiveness.

"Among 74 FDA-registered studies, 31%, accounting for 3449 study participants, were not published," the researchers wrote. "Whether and how the studies were published [was] associated with the study outcome."

"A total of 37 studies [of 12 antidepressant agents] viewed by the FDA as having positive results were published; one study viewed as positive was not published. Studies viewed... as having negative or questionable results were, with 3 exceptions, either not published... or published in a way that, in our opinion, conveyed a positive outcome."

On the basis of the published literature, they said, "94% of the trials [appeared] positive. By contrast, the FDA analysis showed that 51% were positive."

"I spent 3 years at the FDA reviewing studies," lead author Erick H. Turner, MD, who is now assistant professor of psychiatry at the Oregon Health and Science University and medical director of the mood disorders program at the Portland Veterans Affairs Medical Center, told Psychiatric Times.

"I previously had been at [NIMH], the Mecca of psychiatric research. When I got to the FDA, it was like a whole new world. I was reviewing new drug applications, and I kept seeing these negative studies. I had never seen anything like that before. I asked my boss and he just said, 'We see that all the time.' He shrugged and said, 'That's the way it goes.'"

"He knew about it. My coworkers knew about it. People in the industry knew about it. But it was news to me."

Turner was maintaining a private practice while working at the FDA. "It suddenly didn't seem right to me that I was prescribing drugs for my patients, but [that] I never saw any journal articles telling me not to do anything. I realized I was not getting the full picture as a clinician."

After moving to Portland, Turner said, "I felt like [a] door had been slammed shut in my face. I was used to having access to this information. It was like I had been let in on a secret but couldn't tell anyone about it because I couldn't prove it."

He wrote an article for the online journal, PLoS Medicine, calling for the FDA to open its New Drug Application database to allow for public access to test protocols and results.

The National Library of Medicine and the FDA opened a Web-based registry, ClinicalTrials.gov, in 2000 as a result of the FDA Modernization Act of 1997. In an article describing the project in the Journal of the American Medical Informatics Association, the creators wrote, "If relevant data about trials are not published or are poorly reported, then this could lead to severe publication bias and... poor care."

The International Committee of Medical Journal Editors (ICMJE) recognized the issue of publication bias in 2004. In an editorial published simultaneously in 11 journals and on MEDLINE, they wrote, "Honest reporting begins with revealing the existence of all clinical studies, even those that reflect unfavorably on a research sponsor's product." But, they added, "selective reporting does occur, distorting the body of evidence available for clinical decision making.

"When research sponsors or investigators conceal the presence of selected trials, these studies cannot influence the thinking of patients, clinicians, other researchers, and experts who write practice guidelines or decide on insurance-coverage policy."

The solution, the editors wrote, was registration. "If all trials are registered in a public repository at their inception, every trial's existence is part of the public record, and the many stakeholders in clinical research can explore the full range of clinical evidence. We are far from this ideal at present, since trial registration is largely voluntary, registry data... and public access... vary, and registries contain only a small proportion of trials."

ICMJE announced that its members would require studies submitted for publication to be registered with a clinical trials registry, beginning in July 2005. As of 2007, the editors recognized 6 principal registries for clinical trial data (Table).

Turner, however, sees 2 major loopholes in the ICMJE's registration plan. First, he noted that researchers publishing in journals that do not belong to ICMJE are not required to register their studies. Second, "Suppose you have registered it with clinicaltrials. gov and the study doesn't work. There is no requirement to publish it. You can just deep-six it and forget it."

Editors are aware of the drawbacks. "We would relish getting articles about negative studies," said Michael Roy, managing editor of the American Journal of Psychiatry, in an interview. "It's part of the full context that our readers need to know."

But Roy pointed out that his journal publishes only about 7% of the articles submitted to it. Some do not meet the journal's standards, and the ones that do must go through a double-blind peer-review process. If there is a question about the data, the editors can ask for more information.

"The authors have to give us a clinical trial registration number when they submit the article," he said, "so there is some tool that we can use. The editor can and sometimes does ask for the raw data."

Congress called for stricter registration rules in 2005, and passed the FDA Act Amendments of 2007, which strengthened the requirements for registration. All new clinical trials must be registered before patients can be recruited.

But Turner believes the 2007 amendments did not go far enough. "The law calls for registration and makes it mandatory," he said, "but only on a prospective basis. That's great for a drug that's coming through the pipeline right now. But what about all the drugs that we reviewed in this journal article [mentioned above]? They're grandfathered in. Most of these studies were done before the 2005 Act. The FDA had the record of those studies, but no one else did."

Jeffrey M. Drazen, MD, editor-in-chief of the New England Journal of Medicine, was not available to be interviewed for this story. Through a representative, he told Psychiatric Times, "Since there is now a requirement for all trials to be registered [and] mandated by the [FDA] Amendments Act of 2007, there will be a 'complete list' of all trials. The FDA also requires that all data be reported-positive and negative-in a results database openly accessible to the public. This is a solution to the problem of selective awareness."

But Turner would like to see more. "FDA reviews have been going on for decades. They get the protocols before the studies are done. Later, they review the studies, including the raw data used by reviewers. They see the studies that don't get published, so they can be sure that they don't get spun. This was the data I used for this study. These could be made a lot more available than they are."