Small Effects Are Not Trivial From a Public Health Perspective

(Please see Point article by Irving Kirsch, PhD, and David Antonuccio, PhD)

Irving Kirsch, PhD, and David Antonuccio, PhD, point out that antidepressants have small specific effects in placebo-shielded, randomized, controlled trials (RCTs). Similar data from meta-analyses of related data sets have been out there for several years. Our perceptions about antidepressant efficacy have been inflated by the "file drawer" effect: the selective publication of positive studies and suppression of negative ones (Thase, 2002a). In order to properly evaluate efficacy, the meta-analyst must have access to all relevant trials. This became feasible when the Freedom of Information Act was used to obtain the New Drug Application (NDA) data sets from the U.S. Food and Drug Administration (Khan et al., 2000; Walsh et al., 2002). Using the NDA summary reports for six newer drugs (fluoxetine [Prozac], sertraline [Zoloft], paroxetine [Paxil], venlafaxine [Effexor], nefazodone [Serzone] and citalopram [Celexa]), Kirsch et al. (2002) found that antidepressants offer only about a 20% (two-point) greater reduction in Hamilton Rating Scale for Depression (HAM-D) scores than placebo.

There is now no doubt that nonspecific factors (i.e., placebo, expectancy, social support, spontaneous remission) account for a majority of the explainable variance in industry-funded RCTs of outpatients with depression. The same may be true for federally funded studies of similar patient groups (i.e., Elkin et al., 1989; Hypericum Depression Trial Study Group, 2002). To conclude from these data that antidepressants do not have clinically meaningful effects, however, requires overlooking a number of issues.

There are serious methodologic and analytic problems that compromise the design sensitivity of contemporary RCTs of antidepressants (Thase, 2002b). Briefly, the studies are underpowered (mean power

50%), pay too little attention to reliability of assessment, are often overinclusive (i.e., investigators are paid by number enrolled, not number excluded) and are analyzed with outmoded approaches that do not accurately take into account the effects of attrition (Klein et al., 2002; Thase, 2002a). These studies, by and large, do not enroll patients with the subforms of depression that respond more favorably to antidepressants and are plagued by "inflation" of entry symptom severity thresholds (Klein et al., 2002). The problems persist largely because the standard methods still work well enough to accomplish the studies' primary aim of the sponsors: to obtain FDA approval of their medications (Thase, 2002b).

Kirsch and Antonuccio suggest that the "hidden moderators" argument also cannot account for much of an effect. However, if only 10% to 20% of a group of outpatients respond specifically to the drug, a mean two-point advantage would be explained by a 10- to 15-point average difference among the small subgroup's true drug responders. Thus, the one or two additional patients who respond to an active medication (per 10 treated) get substantial benefit. This is not a trivial effect from a public health perspective (Thase, 2002a).

The greater relative benefit of antidepressant therapy among more severely depressed patients (e.g., Elkin et al., 2002, or Thase et al., 1997a) is offhandedly rejected as an artifact or "regression to the mean." Most recently, Khan and colleagues (2002) reported a steady linear relationship between pretreatment severity and the probability of observing a significant drug-placebo difference. The well-replicated observation that milder depressions are more placebo-responsive is not mentioned. Nor is the futility of placebo treatment of psychotic depression.

Kirsch and Antonuccio also minimize the rapid loss of drug effect when antidepressants are withdrawn prematurely. The statement that placebo accounts for 71% of the benefits of continuation-phase therapy hides a difference of 20% in relapse rates (i.e., 50% versus 70% survival). This again is a significant effect. Moreover, although I was unable to review the study of Walach and Maidhof (1999), I believe that the direction of the described effect has been reversed. In the antidepressant continuation-therapy literature, most relapses on placebo occur during the first six months; the curve of an antidepressant-treated group has a flatter slope (Thase, 2000). In other words, a longer study length is more closely related to active drug failure (than is the case with placebo).

A final point of (largely) agreement: psychoeducation, exercise and proven, focused psychotherapies are underutilized as treatments of depression. I wish that all individuals with mild to moderate depression who wanted these interventions received them first, reserving pharmacotherapy for nonresponders (Thase et al., 1997a). But for patients with more severe or chronic depression, there is now good evidence that the combination of pharmacotherapy and psychotherapy is superior to either treatment alone (Keller et al., 2000; Thase et al., 1997b). I suggest that we work harder to ensure that more people with depression get adequate, evidence-based treatments and stop squabbling over the magnitude of specific and nonspecific elements of treatment.

Dr. Thase is professor of psychiatry at University of Pittsburgh Medical Center. He is also director of the depression treatment and research program and chief of adult academic psychiatry at Western Psychiatric Institute.

References:

References1.Elkin I, Shea MT, Watkins JT et al. (1989), National Institute of Mental Health Treatment of Depression Collaborative Research Program. General effectiveness of treatments. Arch Gen Psychiatry 46(11):971-982; discussion, 983 [see comment].
2.Hypericum Depression Trial Study Group (2002), Effect of Hypericum perforatum (St John's Wort) in major depressive disorder: a randomized controlled trial. JAMA 287(14):1807-1814 [see comments].
3.Keller MB, McCullough JP, Klein DN et al. (2000), A comparison of nefazodone, the cognitive behavioral-analysis system of psychotherapy, and their combination for the treatment of chronic depression. [Published erratum N Engl J Med 345(3):232.] N Engl J Med 342(20):1462-1470 [see comments].
4.Khan A, Leventhal RM, Khan SR, Brown WA (2002), Severity of depression and response to antidepressants and placebo: an analysis of the Food and Drug Administration database. J Clin Psychopharmacol 22(1):40-45.
5.Khan A, Warner HA, Brown WA (2000), Symptom reduction and suicide risk in patients treated with placebo in antidepressant clinical trials: an analysis of the Food and Drug Administration database. Arch Gen Psychiatry 57(4):311-317 [see comments].
6.Kirsch I, Moore TJ, Scoboria A, Nicholls SS (2002), The emperor's new drugs: an analysis of antidepressant medication data submitted to the U.S. Food and Drug Administration. Prevention & Treatment 5:Article 23. Accessed Aug. 5.
7.Klein DF, Thase ME, Endicott J et al. (2002), Improving clinical trials: American Society of Clinical Psychopharmacology recommendations. Arch Gen Psychiatry 59(3):272-278 [see comments].
8.Thase ME (2000), Relapse and recurrence of depression. An updated practical approach for prevention. In: Drug Treatment Issues in Depression, Palmer KJ, ed. Auckland, New Zealand: Adis International Limited, pp35-52.
9.Thase ME (2002a), Comparing the methods used to compare antidepressants. Psychopharmacol Bull 36(suppl 1):4-17.
10.Thase ME (2002b), Studying new antidepressants: if there was a light at the end of the tunnel could we see it? J Clin Psychiatry 63(suppl 2):24-28.
11.Thase ME, Buysse DJ, Frank E et al. (1997a), Which depressed patients will respond to interpersonal psychotherapy? The role of abnormal EEG sleep profiles. Am J Psychiatry 154(4):502-509.
12.Thase ME, Greenhouse JB, Frank E et al. (1997b), Treatment of major depression with psychotherapy or psychotherapy-pharmacotherapy combinations. Arch Gen Psychiatry 54(11):1009-1015.
13.Walach H, Maidhof C (1999), Is the placebo effect dependent on time? A meta-analysis. In: How Expectancies Shape Experience, Kirsch I, ed. Washington, D.C.: American Psychological Association, pp321-332.
14.Walsh BT, Seidman SN, Sysko R, Gould M (2002), Placebo response in studies of major depression: variable, substantial, and growing. JAMA 287(14):1840-1847 [see comment].