Antidepressants in Bipolar Depression: A New Meta-Analysis for an Old Controversy

Critics have noted that meta-analysis, when misused, resembles statistical alchemy, taking the dross of individually negative studies to produce the gold of a positive pooled result.² This happened with an influential 2004 publication on antidepressants in acute bipolar depression, and it is being slightly rectified with a recently published update of that study.^3,4 In both cases, reasonable judgments cannot be made unless we understand how to use, and abuse, meta-analysis.

Meta-analysis represents an observational study of studies: one combines results of different studies into one summary measure, weighted for sample size and data variability. These strengths come at a cost: meta-analysis can be invalid if studies differ greatly from each other-the problem of “heterogeneity” (sometimes called the “apples and oranges” problem), which reflects confounding bias, the key scientific problem that our observations are often false because other factors, of which we are not aware, influence the results observed. This is why clinicians should believe only half of what they see, and none of what they hear.⁵ Randomization minimizes confounding bias, since all these other factors will be randomly distributed and thus cancel each other out.

Since meta-analysis is the observational study of studies, it reintroduces confounding bias, and the benefits of randomization are lost. This means that as with any clinical opinion or observational study, we cannot take meta-analyses at face value. The less heterogeneity there is, the more valid the meta-analysis. If studies involve the same drug or same class, in the same clinical setting, with the same research design, then some judgments can be made. Otherwise, most judgments will be false.

The problem of heterogeneity also relates to the number of studies included. Meta-analysis of 2 or 3 studies is practically meaningless. It is like taking symptom scales in 2 or 3 patients and calculating means and standard deviations. Five studies is still borderline. Certainly 10 or more studies would be considered more valid; most good meta-analyses involve dozens of studies. This is not a minor matter; it has been estimated mathematically that if one begins with a low probability (25%) that we believe that something is the case (eg, that antidepressants are effective in bipolar depression), a meta-analysis of a few small studies with much heterogeneity would increase the probability of the outcome (antidepressant efficacy) to about 41% likelihood.⁶ Hardly definitive.

In 2004, the American Journal of Psychiatry unfortunately published just such a study: a meta-analysis of 5 randomized studies of antidepressants in acute bipolar depression. In that report, heterogeneity was rampant. For instance, the only placebo-controlled study that found no evidence of acute antidepressant response is the only study in which all patients received baseline lithium.⁷ That study was excluded from the efficacy analysis on technical grounds. Two studies compared antidepressant alone with placebo alone, without any mood stabilizers; sample sizes in those studies were quite small. One large study drove the whole meta-analysis (N = 433, accounting for 59% of the review sample), and it was an Eli Lilly–conducted study of olanzapine plus fluoxetine versus olanzapine plus placebo⁸; in the meta-analysis, what was called “placebo” was actually olanzapine, whereas in most of the other studies, patients literally got placebo (meaning an inert pill).

Exercising the right of occasional suppression and slight modification, it is truly absurd to see how plastic a limited number of observations become, in the hands of men with preconceived ideas.^1(p267)

– Sir Francis Galton, 1863

Apples and oranges are too weak as metaphors: Mars versus Venus would be better. Of course, antidepressants will “not cause” mania when most patients defined as getting “placebo” are in fact getting an antimanic neuroleptic. The claim of efficacy in 4 totally different studies involves much probable confounding bias. Decent randomized studies are so mixed that the results represent the ullage of what they once were. Such data manipulation may serve demotic purposes of persuasion, but it does not serve the scientific purpose of approximating truth. Unfortunately, like an infection that spreads by citation, this kind of prominently published study has had impact.

The new update basically adds the best-designed largest study to date (N = 332), the STEP-BD study of bipolar depression, in which bupropion or paroxetine was equivalent to placebo when added to standard mood stabilizers.⁹ This confirms the previous similar study excluded from the prior meta-analysis.7 The finding of “no benefit,” when added to the previous tenuous meta-analysis, moves the meta-analytic summary result closer to the null value, hence the “new” conclusion that antidepressants do not work for bipolar depression. Again there was “no” mania, which is not surprising, since the STEP-BD patients were all treated with antimanic medications. Also, the new meta-analysis does not emphasize the clear and repeated finding of inefficacy in a meta-analysis of maintenance treatments, a clinically important matter, separate from whether or not there is any short-term benefit for severe acute depression.¹⁰

What should clinicians take away? They should not have believed the results of the first meta-analysis for the reasons given above. The second one only clarifies the clear flaws of the first. The best sources are good, well-designed, randomized studies, such as the STEP-BD study, which speak clearly: in general, antidepressants are not effective in acute bipolar depression for the average patient. As with any average, there will be exceptions-a minority in whom some utility will be seen. But, this does not justify extensive use of antidepressants. Contrary to occasional claims otherwise, antidepressants are still widely prescribed in bipolar depression, in the US and throughout the world, and are passionately believed in by many experts and clinicians.^11,12 More studies are always welcomed. But for now, it is hard to see how any other conclusion can be made except that the best available scientific evidence contradicts these claims and practices.

References:

References

1.

Stigler S.

The History of Statistics: The Measurement of Uncertainty Before 1900.

Cambridge, MA: Harvard University Press; 1986.

2.

Feinstein AR. Meta-analysis: statistical alchemy for the 21st century.

J Clin Epidemiol

. 1995;48:71-79.

3.

Gijsman HJ, Geddes JR, Rendell JM, et al. Antidepressants for bipolar depression: a systematic review of randomized, controlled trials.

Am J Psychiatry.

2004;161:1537-1547.

4.

Sidor MM, Macqueen GM. Antidepressants for the acute treatment of bipolar depression: a systematic review and meta-analysis.

J Clin Psychiatry.

2010 Oct 5; [Epub ahead of print].

5.

Ghaemi SN. A Clinician’s Guide to Statistics and Epidemiology in Mental Health. Cambridge, UK: Cambridge University Press; 2010.

6.

Ioannidis JP. Why most published research results are false.

PLoS Med.

2005;2:e124.

7.

Nemeroff CB, Evans DL, Gyulai L, et al. Double-blind, placebo-controlled comparison of imipramine and paroxetine in the treatment of bipolar depression.

Am J Psychiatry

. 2001;158:906-912.

8.

Tohen M, Vieta E, Calabrese J, et al. Efficacy of olanzapine and olanzapine-fluoxetine combination in the treatment of bipolar I depression [published correction appears in

Arch Gen Psychiatry

. 2004;61:176].

Arch Gen Psychiatry

. 2003;60:1079-1088.

9.

Sachs GS, Nierenberg AA, Calabrese JR, et al. Effectiveness of adjunctive antidepressant treatment for bipolar depression.

N Engl J Med

. 2007;356:1711-1722.

10.

Ghaemi SN, Wingo AP, Filkowski MA, Baldessarin RJ. Long-term antidepressant treatment of bipolar depression: meta-analyses of benefits and risks.

Acta Psychiatr Scand

. 2008;118:347-356.

11.

Baldessarini RJ, Leahy L, Arcona S, et al. Patterns of psychotropic drug prescription for U.S. patients with diagnoses of bipolar disorders.

Psychiatr Serv

. 2007;58:85-91.

12.

Grunze H, Kasper S, Goodwin G, et al; World Federation of Societies of Biological Psychiatry Task Force on Treatment Guidelines for Bipolar Disorders. World Federation of Societies of Biological Psychiatry (WFSBP) guidelines for biological treatment of bipolar disorders. Part I: treatment of bipolar depression.

World J Biol Psychiatry.

2002;3:115-124.