Debate: What’s the Evidence for the Evidence-Based Treatments of Depression?

POINT

The Specific Efficacies of CBT and IPT Have Not Been Established

In the past few years, large meta-analyses of antidepressant drug treatments for major depression have failed to show distinct advantages over placebo. The authors of one such report provocatively concluded that the new antidepressant drugs are either placebos or act as placebos and therefore are not clinically meaningful. Some authors have argued that such findings advance the utility of the evidence-based nondrug treatments-cognitive-behavioral therapy (CBT) and interpersonal psychotherapy (IPT).¹

But what holds for the goose often holds for the gander. We have undertaken reviews of both CBT and IPT as treatments for major depression.^2,3 When compared with plausible control psychotherapies, they also showed minimal differentiation. Just as some authors claim that the evidence shows that antidepressant drugs are no more than placebos, the same suggestions have been put forth about CBT and IPT. For instance, the psychologist Gary Greenberg^4(p316) advanced the view that CBT is more an ideology and a “method of indoctrination into the pieties of American optimism.”

The key issue then is why do the so-called evidence-based or empirically supported treatments (ESTs) for depression (ie, antidepressant drugs, CBT, IPT) produce this perplexing result when all 3 are invariably positioned in treatment guidelines as the mainstay of treatment for managing depression?

We offer 2 principal reasons here. First, we outline the methodological limitations of psychotherapy efficacy research and the role that this has played in falsely positioning CBT and IPT as first-line or superior treatments for depres-sion. As part of this, we consider what is meant by the placebo effect. Second, we discuss the negative impact of universally applying treatments across nonspecific disorders, such as major depression.

Methodological issues

The supportive evidence base for CBT and IPT as superior treatments for major depression is less substantive than generally considered because of 3 critical methodological flaws in efficacy research.⁵ First, differential effects generally disappear when analyses are limited to therapies that are intended to be therapeutic (bona fide therapies). For example, when meta-analyses specifically removed non–bona fide therapies, CBT did not differ from other psychotherapies for acute-phase treatments of major depression.6 A more recent meta-analysis quantified comparable effects when contrasting IPT (eg, coping-oriented couples therapy, supportive psychotherapy) with other psychotherapies, including CBT for depression-a finding that is consistent with our review of IPT studies.^3,7

Studies or meta-analyses that have shown the superiority of CBT and IPT have generally used benign, ineffective control strategies (eg, wait-list assignment) in randomized controlled trials (RCTs), leading to false positioning of CBT and IPT as superior treatments. Treatment-as-usual comparators also tend to favor the active psychotherapy, because therapist enthusiasm, patient expectancy, and other common factors are less likely to be activated in the treatment-as-usual condition.⁸

For example, the meta-analysis undertaken by Cuijpers and colleagues⁷ reported a moderate to large effect of IPT in the acute treatment of depression compared with other treatment methods. Control groups in RCTs of the effectiveness of psychotherapy consisted of treatment as usual; wait-list; and inert nontherapies, which risked false boosting of IPT effect sizes. Requesting a therapist to be inert (in so-called clinical management) or to provide a psychotherapy with only nonspecific therapeutic ingredients is difficult for any therapist to implement.

We are not asserting that all treatments are equal, but rather that flawed procedures and paradigms may obscure quantification of true treatment efficacy. Nonetheless, the evidence to date does not support the reified status of CBT and IPT as universal treatments for treating clinical depression.

Evaluating psychotherapies (especially CBT and IPT) is made even more difficult by a second key methodological issue-the role of nonspecific therapeutic factors. Evidence of psychotherapy’s benefits that emerge from its specific ingredients (ie, techniques and therapeutic approaches integral to the therapy’s theoretical model) is lacking; meta-analyses have quantified that only an 8% variance in psychotherapeutic outcome was accounted for by such a specific therapeutic component.⁹ Furthermore, dismantling designs have shown that key components of CBT were not responsible for therapeutic benefit, and others have observed that most improvement occurs before the formal introduction of cognitive restructuring techniques.^10,11

Most of the improvement during psychotherapy is therefore likely to be a consequence of nonspecific effects common to all therapies-namely, an emotionally charged confiding therapeutic relationship, a healing setting, a clear rationale, and a treatment procedure believed by both patient and therapist to be restorative.¹² If the impact of the specific psychotherapeutic ingredient is low-with estimates of up to only 15%-this makes it extremely difficult to quantify its specific impact when compared with any other plausible psychotherapy that provides those nonspecific ingredients.¹³ This issue also helps explain why meta-analyses that support CBT and IPT show significant differences only when the control therapy lacks the key nonspecific therapeutic ingredients. Proper evaluation of the effectiveness of CBT and IPT therefore needs to recognize this major methodological limitation.

This general finding is worthy of some clarification. The use of the term “placebo-controlled trials” has a clear connotation-that the nonactive-treatment group receives a placebo or, by definition, an inert treatment. But should we equate the beneficial nonspecific effects of a therapeutic intervention-whether as a component of psychotherapy or of good clinical care when a therapist principally prescribes an antidepressant drug-as necessarily inert? All psychotherapists effectively prescribe themselves and, while we have long recognized the phenomenon of nonspecific and placebo effects, we doubt whether their impact is sufficiently appreciated.

Kirsch¹⁴ reviewed studies of sham surgeries for patients with Parkinson disease, knee surgery, and angina pectoris and rejected the view that placebo treatments are, by definition, inert. As a component of psychotherapy, they provide the integral, curative, nonspecific that may actually activate spontaneous remission and may provide ongoing benefits. We should not be dismissive of these types of effects and should realize that the term “placebo effect” does not capture the reality that optimal, nonspecific, therapeutic ingredients are powerful and not, in fact, inert. We also need to recognize the distinct contribution of placebo effect in RCTs, which makes clarification of the specific effects of each therapy more difficult, especially for the psychotherapies. This issue should be embraced in study designs rather than ignored or summarily dismissed.

The third methodological issue relevant to psychotherapy efficacy research concerns underestimation of the role of therapeutic allegiance. Allegiance is not only strongly associated with outcome (with effect sizes of up to 0.65), but it may also equalize treatment differences.¹⁵ Meta-analyses have attributed almost one-third of the variance in effect sizes to therapeutic allegiance, and therapeutic allegiance also accounts for 69% of the variance in outcome.^16,17 On the basis of these results, others have quantified that 92.5% of the time therapeutic allegiance alone may predict the treatment that will be most successful.¹⁸ Studies that position CBT and IPT as first-line treatments for depression rarely consider this factor.

Universal application of treatments across nonspecific disorders

We are not suggesting that CBT, IPT, or antidepressant drugs are ineffective treatments for major depression-despite their minimal separation from placebo/control therapies in RCTs. Rather, we wish to argue that their specific efficacy has not been established. The obfuscating problem rises from diagnoses such as major depression and dysthymia that are no more than domain names (like major breathlessness) that subsume multiple depressive conditions (with variable biological, psychological, and social causes) and yet are the categorical diagnoses used in RCTs for antidepressants and psychotherapies.

Imagine if a new and very effective anti-asthma drug was tested in a placebo-controlled trial for those with major breathlessness, but only 5% to 10% of the participants actually had asthma. The risk is that the truly effective anti-asthma drug would not emerge as superior to placebo. In essence, because the methodological approach was logically flawed, logically flawed interpretations follow. Similarly, physicians would not tell their patients that they have diabetes and merely decide treatment on that domain diagnosis alone. Rather, patients would have their diabetes subtyped-with a diagnosis of type 1 diabetes treated with insulin and a type 2 diabetes diagnosis addressed with lifestyle changes.

Thus, we argue for abandoning the current nonspecific model that allows treatments to be tested as if they have a nonspecific application for nonspecific disorder categories because this invariably produces nonspecific results.⁵ Clinical depression is not an entity; it comprises a heterogeneous collection of different conditions that have their own propensities to preferentially respond to drugs and/or psychotherapeutic approaches.

Conclusions

By simply adopting a domain diagnosis of major depression, the true treatment validity of CBT and IPT is obscured. Although we acknowledge that CBT and IPT are likely to be specifically effective in some circumstances, the evidence is yet to be demonstrated and will not be until context-specific clinical syndrome RCTs are undertaken that respect the intrinsic rationale of the specific therapeutic approach. The task is to identify when antidepressant drugs, CBT, IPT, and other therapies are primary and salient in addressing causes and when and if they complement each other, as well as when they might best be operationalized sequentially. “What treatment, by whom, is most effective for this individual with that specific problem, and under which set of circumstances.”^19(p111)

We have outlined several reasons why the evidence-based treatments for depression have been falsely positioned as first-line treatments for depression and why the actual evidence of their effectiveness is compromised by faulty diagnostic paradigms and RCT design strategies. Current clinical management of the depressive disorders risks the affective fallacy-evaluating a work by its effects on the reader rather than by the integral strengths of the work itself.²

In this context, the choice between different treatments for each patient often risks resting simply on the professional and personal biases of the patient and therapist. As it currently stands, the literature gives CBT and IPT high-cachet status and risks overselling them as treatments for depression.

There is strong clinical support for the 3 ESTs considered here, and we do not judge antidepressant drugs, CBT, or IPT as ineffective in the sense of acting only as placebos. However, we do not view any one as having universal relevance or application. That is theoretically nonsensical and, unfortunately, so plays out in the clinical arena. The priority is to not view any therapy as having universal application for clinical depression, to respect a “horses for courses” model, and to determine the specific contribution of different treatments to different conditions.

Gordon Parker, AO, MBBS, MD, PhD, DSc
Scientia Professor of Psychiatry
University of New South Wales
Executive Director, Black Dog Institute
Prince of Wales Hospital
Randwick, NSW, Australia

Kathryn Fletcher
Research Assistant, Black Dog Institute

Professor Parker and Ms Fletcher report that they have no conflicts of interest concerning the subject matter of this article.

References

1. Kirsch I, Deacon BJ, Huedo-Medina TB, et al. Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration. PLoS Med. 2008;5:e45.
2. Parker G, Roy K, Eyers K. Cognitive behavior therapy for depression? Choose horses for courses. Am J Psychiatry. 2003;160:825-834.
3. Parker G, Parker I, Brotchie H, Stuart S. Interpersonal psychotherapy for depression? The need to define its ecological niche. J Affect Disord. 2006;95:1-11.
4. Greenberg G. Manufacturing Depression: The Secret History of a Modern Disease. New York: Simon & Schuster; 2010.
5. Parker G, Fletcher K. Treating depression with the evidence-based psychotherapies: a critique of the evidence. Acta Psychiatr Scand. 2007;115:352-359.
6. Wampold BE, Minami T, Baskin TW, Callen Tierney S. A meta-(re)analysis of the effects of cognitive therapy versus ‘other therapies’ for depression. J Affect Disord. 2002;68:159-165.
7. Cuijpers P, Geraedts AS, van Oppen P, et al. Interpersonal psychotherapy for depression: a meta-analysis. Am J Psychiatry. 2011;168:581-592.
8. Westen D, Novotny CM, Thompson-Brenner H. EBP ≠ EST: Reply to Crits-Christoph et al. (2005) and Weisz et al. (2005). Psychol Bull. 2005;131:427-433.
9. Wampold BE. The Great Psychotherapy Debate: Models, Methods, and Findings. Mahwah, NJ: Lawrence Erlbaum Associates; 2001.
10. Jacobson NS, Dobson KS, Truax PA, et al. A component analysis of cognitive-behavioral treatment for depression. J Consult Clin Psychol. 1996;64:295-304.
11. Ilardi SS, Craighead WE. The role of nonspecific factors in cognitive-behavior therapy for depression. Clin Psychol Sci Pract. 1994;1:138-156.
12. Frank JD. Common features of psychotherapy. Aust N Z J Psychiatry. 1972;6:34-40.
13. Lambert MJ. Implications of outcome research for psychotherapy integration. In: Norcross JC, Goldstein MR, eds. Handbook of Psychotherapy Integration. New York: Basic Books; 1992:94-129.
14. Kirsch I. The Emperor’s New Drugs: Exploding the Antidepressant Myth. New York: Basic Books; 2010.
15. Luborsky JL, Barrett MS. What else materially influences what is represented and published as evidence? In: Norcross JC, Beutler LE, Levant RF, eds. Evidence-Based Practices in Mental Health: Debate and Dialogue on the Fundamental Questions. Washington, DC: American Psychological Association Press; 2005:257-267.
16. Robinson LA, Berman JS, Neimeyer RA. Psychotherapy for the treatment of depression: a comprehensive review of controlled outcome research. Psychol Bull. 1990;108:30-49.
17. Luborsky L, Diguer L, Seligman DA, et al. The researcher’s own therapy allegiances: a “wild card” in comparisons of treatment efficacy. Clin Psychol Sci Pract. 1999;6:95-106.
18. Westen D, Novotny CM, Thompson-Brenner H. The empirical status of empirically supported psychotherapies: assumptions, findings, and reporting in controlled clinical trials. Psychol Bull. 2004;130:631-63.
19. Paul GL. Strategy of outcome research in psychotherapy. J Consult Psychol. 1967;31:109-118.

(See Counterpoint by Steven D. Hollon, PhD)

COUNTERPOINT

Is Specificity Relevant When Treating Depression With Evidence-Based Treatments?

Dr Parker and Ms Fletcher¹ argue that specificity has yet to be established for cognitive-behavioral therapy (CBT) or interpersonal psychotherapy (IPT)-the two psychosocial interventions most often touted as evidence-based treatments for depression. I agree with some of the points the authors make, but I disagree with others.

First, a distinction needs to be made between specificity and superiority. I agree with Parker and Fletcher (and Wampold before them) that nonspecific psychotherapy controls are hard to implement in a convincing manner. To the extent that is true, then finding a psychosocial intervention superior to a nonspecific psychosocial control could simply reflect a more powerful mobilization of those nonspecific factors (“my nonspecifics are bigger than your nonspecifics”). Therefore, one could find superiority in the absence of specificity.

The same concern can be raised with respect to comparisons between psychotherapies. As Parker and Fletcher also point out (and Luborsky before them), allegiance effects loom large in the treatment literature and account for much of the variance in outcomes. To the extent that allegiance effects reflect differences in enthusiasm for one approach over the other, then one could again find superiority in the absence of specificity. However, differential allegiance could also reflect differences in competence to implement the respective interventions.² Such comparisons depend on the capacity of the research team to implement each of the respective interventions in a competent fashion, and it is rare that any one research group is competent to implement multiple psychotherapies in the same trial.

These concerns do not extend to pill-placebo controls in the context of comparisons to medications, since such controls can be implemented in a blinded and relatively unbiased fashion and medication efficacy depends less on the skill of the provider than on the dosage provided. Most such trials have found psychotherapy, including CBT for more severe or atypical depression, IPT or behavioral activation (BA) for more severe depression, and problem-solving therapy in general practice, to be superior to placebo and comparable to antidepressant medications.^3-7

CBT was no better than pill-placebo and less effective than antidepressants in patients with more severe depression in the studies by Elkin and associates5 and Dimidjian and colleagues.⁶ However, questions can be raised about the quality with which CBT was implemented in the first study and the role of allegiance effects in the second (the Dimidjian study was conducted at the home of BA).⁸ Results with dynamic psychotherapy did not differ from those with pill-placebo in the only such comparison to date, but neither did antidepressant medication and the two psychotherapies differ.⁹ Other types of psychotherapies have yet to be compared with antidepressants.

What these studies suggest in aggregate is that CBT, IPT, and perhaps BA may have specific effects when competently implemented, but only for patients with more severe depression. In fact, among studies that tested for moderation with respect to severity, specific effects were found only among patients with more severe depression, with respect to both psychotherapy and medications.^10-13 The latter is the “dirty little secret” of a pharmaceutical industry that has long selectively screened out patients with less severe depression (in order to up the odds of finding drug-placebo differences required to win FDA approval) and then turns around and markets those same medications to people who it knows full well are likely to respond for purely psychological reasons.¹⁴

What all this means is that nonspecific factors are likely to be sufficient to produce response in most patients with less severe depression (the majority of patients with major depressive disorder) and that treatments with specific effects are required for only those patients with more severe (and possibly chronic or comorbid) depression. Virtually every psychosocial intervention that has been tested has been shown to be better than its absence, and Parker and Fletcher are correct that meta-analytic reviews suggest that differences are few among them and that nonspecific factors account for the largest portion of the change.^15,16 When practice guidelines privilege CBT and IPT in their recommendations, it is because they have been tested more extensively and against more rigorous comparators, such as medication treatment, not because they have been shown to be superior to other types of psychotherapies in head-to-head comparisons.

Two other factors that likely influence guideline recommendations that Parker and Fletcher do not address are the greater breadth of effect shown by IPT and the more enduring effects shown by CBT relative to medications. There are indications that IPT is not only as effective as antidepressants in reducing acute distress but is also better at addressing relationship problems.^17,18 CBT and perhaps BA have enduring effects not found for antidepressant medications that protect against relapse following treatment termination.^19,20 Parker and Fletcher do not address these benefits, but rather treatment guidelines.

Absence of evidence is not evidence of absence. Just because something has not been tested does not mean that it does not work. However, when third parties are making decisions regarding treatment selection, absence of evidence is tantamount to absence of ticket. Guideline generators and policy makers look to the preponderance of the evidence. CBT and IPT have the most good-quality evidence of any of the existing psychotherapies to support their utility. This does not necessarily mean that their specificity (with respect to common factors) has been established (although the placebo-controlled trials certainly speak to that issue) or that their superiority to other psychotherapies has been established as well.

When other psychotherapies are not recommended by guidelines or selected by third-party payers, it is not because they have failed in direct comparisons with CBT or IPT (by and large those comparisons do not exist) but because they have not been tested as often or as stringently. Parker and Fletcher are right to call attention to that fact and it would be unwise to overstate the case for CBT or IPT (or especially BA, which has yet to be extensively tested). At the same time, it is not unreasonable for the recent guidelines to have greater confidence in CBT or IPT than they do for other approaches.

Steven D. Hollon, PhD
Professor
Department of Psychology
Vanderbilt University
Nashville, Tenn

Dr Hollon reports that he is the recipient of the NIH/NIMH P01 MH60713 grant.

References

1. Parker G, Fletcher K. Treating depression with the evidence-based psychotherapies: a critique of the evidence. Acta Psychiatr Scand. 2007;115:352-359.
2. Leykin Y, DeRubeis RJ. Allegiance in psychotherapy outcome research: separating association from bias. Clin Psychol Sci Pract. 2009;16:54-65.
3. DeRubeis RJ, Hollon SD, Amsterdam JD, et al. Cognitive therapy vs medications in the treatment of moderate to severe depression. Arch Gen Psychiatry. 2005;62:409-416.
4. Jarrett RB, Schaffer M, McIntire D, et al. Treatment of atypical depression with cognitive therapy or phenelzine: a double-blind, placebo-controlled trial. Arch Gen Psychiatry. 1999;56:431-437.
5. Elkin I, Shea MT, Watkins JT, et al. National Institute of Mental Health Treatment of Depression Collaborative Research Program. General effectiveness of treatments. Arch Gen Psychiatry. 1989;46:971-982.
6. Dimidjian S, Hollon SD, Dobson KS, et al. Behavioral activation, cognitive therapy, and antidepressant medication in the acute treatment of major depression. J Consult Clin Psychol. 2006;74:658-670.
7. Mynors-Wallace LM, Gath DH, Lloyd-Thomas AR, Tomlinson D. Randomized controlled trial comparing problem solving treatment with amitriptyline and placebo for major depression in primary care. BMJ. 1995;310:441-445.
8. Jacobson NS, Hollon SD. Prospects for future comparisons between drugs and psychotherapy: lessons from the CBT-versus-pharmacotherapy exchange. J Consult Clin Psychol. 1996;64:104-108.
9. Barber JP, Barrett M, Gallop R, et al. Short-term psychodynamic therapy vs. pharmacotherapy for major depressive disorders: the role of minority status and gender. J Clin Psychiatry. In press.
10. Driessen E, Cuijpers P, Hollon SD, Dekker JJ. Does pretreatment severity moderate the efficacy of psychological treatment of adult outpatient depression? A meta-analysis. J Consult Clin Psychol. 2010;78:668-680.
11. Fournier JC, DeRubeis RJ, Hollon SD, et al. Antidepressant drug effects and depression severity: a patient-level meta-analysis. JAMA. 2010;303:47-53.
12. Khan A, Leventhal RM, Khan SR, Brown WA. Severity of depression and response to antidepressants and placebo: an analysis of the Food and Drug Administration database. J Clin Psychopharmacol. 2002;22:40-45.
13. Kirsch I, Deacon BJ, Huedo-Medina TB, et al. Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration. PLoS Med. 2008;5:e45.
14. Kirsch I. The Emperor’s New Drugs: Exploding the Antidepressant Myth. New York: Basic Books; 2010.
15. Cuijpers P, van Straten A, Warmerdam L, Smits N. Characteristics of effective psychological treatments of depression: a metaregression analysis. Psychother Res. 2008;18:225-236.
16. Cuijpers P, van Straten A, Andersson G, van Oppen P. Psychotherapy for depression in adults: a meta-analysis of comparative outcome studies. J Consult Clin Psychol. 2008;76:909-922.
17. Weissman MM, Klerman GL, Paykel ES, et al. Treatment effects on the social adjustment of depressed patients. Arch Gen Psychiatry. 1974;30:771-778.
18. Weissman MM, Klerman GL, Prusoff BA, et al. Depressed outpatients. Results one year after treatment with drugs and/or interpersonal psychotherapy. Arch Gen Psychiatry. 1981;38:51-55.
19. Dobson KS, Hollon SD, Dimidjian S, et al. Randomized trial of behavioral activation, cognitive therapy, and antidepressant medication in the prevention of relapse and recurrence in major depression. J Consult Clin Psychol. 2008;76:468-477.
20. Hollon SD, DeRubeis RJ, Shelton RC, et al. Prevention of relapse following cognitive therapy vs medications in moderate to severe depression. Arch Gen Psychiatry. 2005;62:417-422.