STAR*D’s Cumulative Remission Rate and Why It Still Matters

Choi_ Nikolai_AdobeStock

COMMENTARY

The well-written Psychiatric Times article concluded, “it is urgent for the field of psychiatry to reconcile the significant differences in remission rates for patients with [major depressive disorder; MDD] as published in the original STAR*D article in 2006 with the reanalysis just published in the BMJ article this year.”¹ And we agree.

As was noted, if our reanalysis of STAR*D’s patient-level data set is correct—that STAR*D’s cumulative remission rate was only approximately half of that reported in 2006—this is a major setback for psychiatry, “as all of the publications and policy decisions based on the STAR*D findings that became clinical dogma since 2006 will need to be reviewed, revisited, and possibly retracted.”¹

Coincidentally, the December 2023 issue of the American Journal of Psychiatry published a letter by the STAR*D primary investigators (PIs) titled, “The STAR*D Data Remain Strong: Reply to Pigott et al” in which they double down on their claim of a 67% cumulative remission rate after up to 4 trials of antidepressant therapies.²

STAR*D PIs alleged that we applied post hoc criteria to selectively eliminate data from 941 patients in our reanalysis of STAR*D’s patient-level data set obtained from the National Institute of Mental Health (NIMH).³

While providing a seemingly erudite defense of their 67% remission rate claim, STAR*D PIs failed to acknowledge:

• Our reanalysis was funded by the Restoring Invisible and Abandoned Trials initiative.
• We published a Call to Action in BMJ documenting our concerns regarding protocol violations.⁴
• We gave STAR*D PIs fair warning of our intentions and inquired whether they would undertake the reanalysis, this time adhering to the NIMH-approved protocol; they declined.
• Prior to publication, STAR*D PIs were given the opportunity to respond to our reanalysis of their work. As BMJ Open’s editor noted, “We invited the authors of the STAR*D study to provide a response to this article, but they declined.”⁵
• As documented, our reanalysis adhered to the research protocol, and where the protocol was silent we used other STAR*D publications to guide our analyses—this is the exact opposite of applying “post hoc” criteria to selectively eliminate patient data, as Rush et al claim.

This methodology discovered several scientific errors (Table).^6,7

Table. STAR*D Scientific Errors^6,7

Sanitized Emergent Suicidality Data

This failure to disclose data for patients who were prescribed citalopram and dropped out is of particular concern. Whether by happenstance or design, this omission both inflated STAR*D’s reported remission rate while circumventing investigators’ obligation to report on emergent suicidality when these 234 patients were most vulnerable.

Research indicates patients prescribed an antidepressant are far more likely to become suicidal earlier—vs later—during treatment.⁸ Although STAR*D’s step 1 article declared “There were no suicides in the 2876 participants in this acute-phase citalopram study,”⁹ this declaration excluded from analysis the 234 patients on citalopram when they were most at risk.

STAR*D PIs published an analysis of emergent suicidality in 2009, stating, “One of the participants who did not return after baseline was reported as a likely suicide secondary to ‘falling’ off his apartment complex roof 10 days after citalopram was prescribed.”¹⁰ However, the article did not report on the status of the other 233 patients on citalopram who dropped out. This scientific error resulted in the underreporting of emergent suicidality/completed suicides in step 1. At a minimum, it failed to report 1 apparent suicide.

Our Analysis for Missing Exit HRSDs

STAR*D PIs correctly stated that we followed their analytic plan, noting “our primary analyses classified patients with missing exit [Hamilton Rating Scale for Depression] HRSD scores as nonremitters a priori.”⁹

However, in yet another omission, they failed to acknowledge that we also mapped the final QIDS-SR (Quick Inventory of Depressive Symptomatology-Self Rated) score to the HRSD for the 1330 patients missing an exit HRSD,¹¹ and then reported the cumulative remission rate 2 ways: (1) 35.0% when using the protocol-specified HRSD, and (2) 41.3% after adding 195 QIDS-defined remissions for patients missing their exit HRSD.

As we stated, “the 41.3% cumulative remission rate should be viewed as the ‘best-case scenario’ since it added an additional 195 QIDS-defined remissions (a remission measure not specified in the protocol) from the 1330 patients with missing exit HRSD scores.”⁸

Comparing STAR*D’s Rates

To bolster their 67% cumulative remission rate claim, STAR*D PIs cited the role of our co-PI, Jay Amsterdam, MD, in a NIMH-funded study that found a 60% cumulative remission rate after 12 months of antidepressant treatment.¹² They then concluded by stating, “a result that is much closer to the 67% remission rate of the original Rush et al STAR*D report than the Pigott et al rate of 35%.”

Once again, STAR*D PIs have made a scientific error with their apples-to-oranges comparison. Instead, they should have compared STAR*D’s outcomes with its precursor study: the Texas Medication Algorithm Project (T-MAP) Depression Study.¹³ Sharing the same PIs, STAR*D’s treatment methodology closely mirrored T-MAP’s, which reported a 12-month cumulative remission rate of only 11%, despite patients going through their “7-step medication algorithm for nonpsychotic MDD”¹³ protocol vs STAR*D’s 4-step treatment protocol.

The title of the T-MAP study ends with: “A Benchmark for Subsequent Studies.” The investigators should be pleased that they beat their T-MAP benchmark study despite going from a 7-step to a 4-step 12-month-long treatment protocol. Unfortunately, despite Herculean efforts, STAR*D PIs did not beat the benchmark by 600%, as they persist in claiming for STAR*D.

There is no statute of limitations for retracting peer-reviewed articles.¹⁴ Medical journal editors are duty-bound to retract articles when their “findings are no longer considered trustworthy due to scientific misconduct or error.”¹⁵ Such is the case with STAR*D. Retraction is a necessary step to restore trust in the scientific literature and advance the evidence-based treatment of major depression.

Dr Pigott is a licensed psychologist trained as a scientist-practitioner. Mr Kim is a doctoral candidate of psychology at the University of Pennsylvania and clinical psychology resident at Weill Cornell Medicine, New York, New York. Dr Xu is an assistant professor of psychology at the College of Idaho. Dr Kirsch is associate director of the Program in Placebo Studies and lecturer in medicine at the Harvard Medical School and Beth Israel Deaconess Medical Center, both in Boston, Massachusetts. He is also professor emeritus of psychology at the University of Plymouth and University of Hull, both in England, and University of Connecticut in Storrs. Dr Amsterdam is professor emeritus of psychiatry and director of the Depression Research Unit at Perelman School of Medicine, University of Pennsylvania, in Philadelphia.

References

1. Miller JJ. STAR*D dethroned? Psychiatric Times. 2023;40(12).

2. Rush AJ, Trivedi M, Fava M, et al. The STAR*D data remain strong: reply to Pigott et al. Am J Psychiatry. 2023;180(12):919-920.

3. Pigott HE, Kim T, Xu C, et al. What are the treatment remission, response and extent of improvement rates after up to four trials of antidepressant therapies in real-world depressed patients? A reanalysis of the STAR*D study’s patient-level data with fidelity to the original research protocol. BMJ Open. 2023;13(7):e063095.

4. Pigott HE, Dubin W, Kirsch I, et al. Call to action: RIAT reanalysis of the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) study. BMJ. March 6, 2019. Accessed February 1, 2024. https://www.bmj.com/content/346/bmj.f2865/rr-10

5. Branch-Hollis A. Note from the editor team at BMJ Open. BMJ Open. July 26, 2023. Accessed February 1, 2024. https://bmjopen.bmj.com/content/13/7/e063095.responses#note-from-the-editor-team-at-bmj-open

6. Rush AJ, Fava M, Wisniewski SR, et al; STAR*D Investigators Group. Sequenced treatment alternatives to relieve depression (STAR*D): rationale and design. Control Clin Trials. 2004;25(1):119-142.

7. Sequenced Treatment Alternatives to Relieve Depression (STAR*D) Research Protocol. National Institute of Mental Health; 2002.

8. Jick H, Kaye JA, Jick SS. Antidepressants and the risk of suicidal behaviors. JAMA. 2004;292(3):338-343.

9. Trivedi MH, Rush AJ, Wisniewski SR, et al; STAR*D Study Team. Evaluation of outcomes with citalopram for depression using measurement-based care in STAR*D: implications for clinical practice. Am J Psychiatry. 2006;163(1):28-40.

10. Zisook S, Trivedi MH, Warden D, et al. Clinical correlates of the worsening or emergence of suicidal ideation during SSRI treatment of depression: an examination of citalopram in the STAR*D study. J Affect Disord. 2009;117(1-2):63-73.

11. Inventory of Depressive Symptomatology (IDS) and Quick Inventory of Depressive Symptomatology (QIDS). IDS/QIDS. Accessed February 1, 2024. http://ids-qids.org

12. Hollon SD, DeRubeis RJ, Fawcett J, et al. Effect of cognitive therapy with antidepressant medications vs antidepressants alone on the rate of recovery in major depressive disorder: a randomized clinical trial. JAMA Psychiatry. 2014;71(10):1157-1164.

13. Rush AJ, Trivedi MH, Carmody TJ, et al. One-year clinical outcomes of depressed public sector outpatients: a benchmark for subsequent studies. Biol Psychiatry. 2004;56(1):46-53.

14. Whitaker R. The STAR*D scandal: scientific misconduct on a grand scale. Mad in America. September 9, 2023. Accessed February 1, 2024. https://www.madinamerica.com/2023/09/the-stard-scandal-scientific-misconduct-on-a-grand-scale/

15. Fang FC, Casadevall A. Retracted science and the retraction index. Infect Immun. 2011;79(10):3855-3859.