STAR*D: It’s Time to Atone and Retract

FROM OUR READERS

Modern medicine refers to practicing medicine based on evidence derived from sound scientific research. To be an excellent physician, one must perhaps master the art of medicine—but to be a good physician, one must know the science of medicine.

Since the 19th century, psychiatry as a medical specialty has, at times, struggled to meet the scientific rigor expected. This is in part due to the complicated nature of the system studied by the psychological sciences and the difficult task of understanding that system when disease sends it into disarray.

Therefore, when a scientific study does arrive on the scene and sheds light on the complicated task of choosing the best of a plethora of treatments for our patients, we flock to that study like moths to a flame. We hold these studies in high acclaim, use them to teach residents, and refer to them when justifying our treatment decisions. The 3 most important such studies in the recent history of psychiatry are CATIE, STEP-BD, and STAR*D, all of which were published in close succession between 2005 and 2007.

The CATIE trial followed about 1500 patients with schizophrenia who had been treated with a variety of antipsychotics for a period of about 1-and-a-half years. It found that in a naturalistic setting, 74% of patients discontinued their medications. The CATIE article implied that prescribing olanzapine may lead to better compliance by writing that, “olanzapine was the most effective in terms of the rates of discontinuation.” However, a closer reading reveals that no 1 antipsychotic was better than another—that “the difference was not significant after adjustment for multiple comparisons.”¹

The STEP-BD trial followed about 175 patients with bipolar depression who had been treated with mood stabilizers with or without the addition of an antidepressant for about 6 months. The study found that, contrary to popular belief, the addition of antidepressants was not nocive in that it did not cause switches into mania. The study also found that antidepressants, contrary to common practice, were not effective in improving bipolar depression.²

Other studies (although not usually large randomized controlled trials) have disputed the conclusion that antidepressants do not trigger manic episodes and suggested that STEP-BD did not have enough patients or time to prove its point.³

STAR*D is likely the most important of the 3 studies. STEP-BD and CATIE focused on disorders with a prevalence of about 1%, according to the last edition of Kaplan and Sadock.⁴ However, STAR*D focused on depression, which is considered to have a much higher prevalence of about 11%. By 2030, depression is estimated to be responsible for about 5% of all disability, only following HIV/AIDS.⁴

STAR*D also looked at the use of antidepressants in a fairly naturalistic setting, rendering it applicable to many Americans. In a study examining the prescriptions of 40 million Americans, 12% indicated being prescribed antidepressants.⁵ STAR*D is arguably 10 times more relevant than CATIE and STEP-BD.

STAR*D and its Impact

STAR*D (Sequenced Treatment Alternatives to Relieve Depression) was a trial of about 4000 patients with depression. The study was set in a naturalistic setting of primary care and outpatient psychiatry practices, “diverse care settings… so that results could be generalized to a broad group of real-world patients.”⁶

Similarly to many modern clinical practices, a patient with depression was first started on an SSRI (in this case, citalopram) for 3 months. If a patient did not improve, they were advanced through a series of trials of either augmentation with an additional treatment or switching to another treatment altogether; these alternative treatments included both antidepressants as well as therapy.

Patients had an opportunity to advance through 4 or 5 steps of treatment options to achieve remission. Additional measures were taken to render the study more realistic to common practices. Patients were given some say in whether they were offered augmentation, switching, or other combinations of treatment options. Dosing was flexible based on response and adverse effects. Placebo was not an available intervention. In many ways, one could say that STAR*D was a test for whether the modern common treatment of depression works.

The results of STAR*D are familiar to all psychiatrists as commonly tested on boards and residency training exams. According to its abstract, the main conclusion of STAR*D was that, “the overall cumulative remission rate was 67%.”⁷

After 4 to 5 rounds of mostly antidepressants trials, over a period of about 1 year, two-thirds of patients had improved enough to be considered in remission. The other main publicized finding of STAR*D was that each new step had a lower chance of being effective than the one prior. Later attempts at treating depression have “lower acute remission rates” and “higher relapse rates.”

The impact of STAR*D was outstanding; it is highly cited in our textbooks. The latest edition of Tasman’s textbook of psychiatry makes 54 references to the STAR*D trial and includes a full chart of the study.⁸ This is in comparison to 37 references to CATIE and 4 references to STEP-BD.

Similarly, the latest edition of the widely read Maudsley’s prescribing guide makes 9 references to STAR*D in comparison to 6 for CATIE and 1 for STEP-BD.⁹ More than 100 peer-reviewed articles have been written about STAR*D,¹⁰ and the original paper was cited 207 times on PubMed.org in 2023—the most of any year.

Newspapers have regularly promoted its findings and continue to do so. As recently as last year, the New York Times was citing STAR*D as “the largest study of antidepressants to date” and touting its results that more “than 60 percent of those patients actually had a very good response.”

Some may think finding that antidepressants are effective in 67% of patients is trivial; however, the efficacy of antidepressants was not as widely accepted prior to STAR*D. The saturation of psychiatric textbook with STAR*D, more than ever before, solidified that teaching.

In 1980, the case of Ray Osheroff cornered psychiatrists into the fear of a lawsuit if not prescribing antidepressants to patients with prolonged depression, but it did not—as STAR*D did—convince us that antidepressants were particularly effective. For almost 20 years now, STAR*D has been taught to psychiatrists as the proof that after multiple trials, about two-thirds of the time, antidepressants are effective.

Pigott, et al¹¹

That antidepressants are less effective than their name would suggest is not a novel idea. Two years after the publication of STAR*D, Kirsch, et al, published a seminal paper arguing that there is “little evidence to support the prescription of antidepressant medication to any but the most severely depressed.”¹²

However, STAR*D was such a large trial that was orchestrated by some of our most recognized peers, so widely taught and referenced, and published in our leading journal, that it commanded authority. Not until the last few months, since the publishing of Pigott’s article reanalyzing the data of STAR*D,¹¹ has this authority come into serious question. Following is a summary of some of problems levied by Pigott and his team against STAR*D.

The STAR*D results inappropriately handled dropouts, in particular the 370 patients who dropped out before the first follow-up. Contrary to STEP-BD and CATIE, which included all patients, the STAR*D investigators did not include patients who did not attend follow-up visits after being enrolled and even prescribed medication.

Not only were many of those patients dismissed, but the main finding of 67% remission is based on a highly unusual “theoretical” supposition that if those patients had stayed, they would have responded at the same rate as patients who did not drop out.

Compare this approach with the CATIE trial, in which treatment discontinuation was not a marker for remission, but rather considered an unfavorable outcome, as is convention. Were this standard metric applied to STAR*D, it alone would have reduced the remission rate to 51%. In their response to Pigott,¹³ it is notable that the STAR*D investigators did not directly address this concern.

Another issue was the use of the QIDS-SR, or the Quick Inventory of Depressive Symptomatology (self-report), rather than HAM-D (the Hamilton Rating Scale for Depression, also referred to as a HSRD) to measure results. Both scales are similar and resemble the DSM criteria for depression. The change appears to the common reader of the seminal STAR*D article as innocuous.

However, in the study, the QIDS was not blinded, whereas the HAM-D was. It is bizarre to not use the results of a blind assessment when they are available. This is also inconsistent with STEP-BP and CATIE, which both used blinded assessment for results. This change alone would reduce the remission rate of each step of the trial by 3% to 11%. In their response, the STAR*D investigators claim that the non-blinded assessment by the patients themselves was “more accurate” and somehow “specifically requested” by the journal itself.

In our opinion, the gravest claim made by Pigott and his team was that 931 patients who were included in the trial did not have documented evidence of depression by blind assessment (using the HAM-D). They noted that:

• 99 patients scored so low on the HAM-D they did not qualify as having depression at all.
• 508 patients had HAM-D scores low enough to only be considered mildly depressed and were, therefore, not appropriate for a clinical trial.
• 324 patients simply were not measured with the HAM-D prior to the trial.

In their response, the STAR*D investigators claim that the choice by Pigott to remove those patients in his reanalysis was “post-hoc” and “selective.” We are of the opinion that not including patients without depression in a trial of treatment for depression is neither post-hoc or selective, but rather a common example of appropriate inclusion criteria.

Concluding Thoughts

Up to this point, the explanations given for the problems brought forth by Pigott remain entirely insufficient. Many attempts have been made to minimize the impact these problems pose to our field and our patients. Some argue that STAR*D is too old and that there are many new pharmacological options since its publication.¹⁴

It is our opinion that the importance of STAR*D and its ramifications for the field of psychiatry are too serious to be dismissed. STAR*D is too cited and used too often to justify current prescribing practices.

The esteem held by our field in the age of modern medicine rests on the validity of our scientific pursuits. The direction our field has historically taken too often followed the dictates of dogma rather than evidence. We should not continue to make this mistake.

Even beyond the academic realm, psychiatry has faced more scrutiny from the public than any other medical field. Some of this criticism has been unjustified, but much of it has been well earned. The best defense of our field in this arena seems to rest in holding ourselves to the highest standards of integrity.

Nicolas Badre, MD

Jason Compton, MD

Lastly, and most importantly, we have an ethical duty to our patients to take an honest look at the evidence derived from our research when making decisions that will impact their mental health. Our patients, our field, and our integrity demand a better explanation of what happened in STAR*D than what has thus been provided. Short of this, the best remaining course to take is a retraction.

Dr Badre is a clinical and forensic psychiatrist in San Diego. He teaches medical education, psychopharmacology, ethics in psychiatry, and correctional care.

Dr Compton is a member of the psychiatry faculty at the University of California, San Diego. His background includes medical education, mental health advocacy, work with underserved populations, and brain cancer research. Neither author reports any disclosures or conflicts of interest.

References

1. Lieberman JA, Stroup TS, McEvoy JP, et al. Effectiveness of antipsychotic drugs in patients with chronic schizophrenia [published correction appears in N Engl J Med. 2010 Sep 9;363(11):1092-3]. N Engl J Med. 2005;353(12):1209-1223.

2. Sachs GS, Nierenberg AA, Calabrese JR, et al. Effectiveness of adjunctive antidepressant treatment for bipolar depression. N Engl J Med. 2007;356(17):1711-1722.

3. Leverich GS, Altshuler LL, Frye MA, et al. Risk of switch in mood polarity to hypomania or mania in patients with bipolar depression during acute and continuation trials of venlafaxine, sertraline, and bupropion as adjuncts to mood stabilizers. Am J Psychiatry. 2006;163(2):232-239.

4. Boland R, Verdiun M, Ruiz P. Kaplan & Sadock’s Synopsis of Psychiatry. 12th Ed. Lippincott Williams & Wilkins; 2021.

5. Moore TJ, Mattison DR. Adult utilization of psychiatric drugs and differences by sex, age, and race [published correction appears in JAMA Intern Med. 2017 Mar 1;177(3):449]. JAMA Intern Med. 2017;177(2):274-275.

6. Questions and answers about the NIMH Sequenced Treatment Alternatives to Relieve Depression (STAR*D) study—all medication levels. National Institute of Mental Health. November 2006. Accessed March 4, 2024. https://www.nimh.nih.gov/funding/clinical-research/practical/stard/allmedicationlevels

7. Rush AJ, Trivedi MH, Wisniewski SR, et al. Acute and longer-term outcomes in depressed outpatients requiring one or several treatment steps: a STAR*D report. Am J Psychiatry. 2006;163(11):1905-1917.

8. Tasman A, Kay J, Lieberman JA, et al. (Eds). Psychiatry. 4th Ed. Wiley-Blackwell; 2015.

9. Taylor DM, Barnes TRE, Young AH. The Maudsley Prescribing Guidelines in Psychiatry. 14th Ed. Wiley-Blackwell; 2021.

10. Miller J. STAR*D Dethroned? Psychiatric Times. 2023;40(12).

11. Pigott HE, Kim T, Xu C, Kirsch I, Amsterdam J. What are the treatment remission, response and extent of improvement rates after up to four trials of antidepressant therapies in real-world depressed patients? a reanalysis of the STAR*D study's patient-level data with fidelity to the original research protocol. BMJ Open. 2023;13(7):e063095.

12. Kirsch I, Deacon BJ, Huedo-Medina TB, et al. Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration. PLoS Med. 2008;5(2):e45.

13. Rush AJ, Trivedi M, Fava M, et al. The STAR*D data remain strong: reply to Pigott et al. Am J Psychiatry. 2023;180(12):919-920.

14. Aftab A. A new reanalysis of STAR*D data: how does it inform our current understanding of antidepressants? Psychiatry at the Margins. September 23, 2023. Accessed March 4, 2024. https://www.psychiatrymargins.com/p/a-new-reanalysis-of-stard-data