DSM-5 Won’t Solve the Overdiagnosis Problem-But Clinicians Can

Exploring treatment options such as N-acetylcysteine and bipolar-specific psychotherapies is on the horizon for this bipolar series. But with DSM-5 about to arrive, one more examination of bipolar diagnosis is warranted. After all, if a diagnosis is inaccurate, treatment efforts, however well-intentioned, may misfire.

The formal changes in DSM-5 will not be official or public until this month. But the bipolar disorder workgroup proposed 4 changes on the DSM-5 Web site (since removed in anticipation of publication).¹ Two of the proposed changes will tighten bipolar diagnostic requirements, and two will loosen them.

The tightening criteria are (1) diagnosis of hypomania or mania will now require a finding of increased energy along with the rest of the unchanged criteria and (2) bipolar not otherwise specified (BP NOS) becomes bipolar not elsewhere classified (BP NEC), with tighter definitions of subthreshold bipolar variations. The loosening criteria are (1) mixed states will encompass a spectrum of admixtures of manic and depressive symptoms, down to specific minimums and (2) antidepressant-induced hypomania or mania will now qualify a patient for a diagnosis of bipolar disorder.

Will the tighter criteria help address the reported overdiagnosis problem?² Take the fictional case of 31-year-old Ms Alvarez, who, since age 18, has had many clearly recognizable episodes of depression that last a week or more. When asked, she endorses phases of increased activity and remarkable productivity; increased social interaction; and a confident, positive outlook that is otherwise unusual for her-all on about 4 to 5 hours of sleep. She does not endorse increased energy during these phases, however.

Does she have bipolar II? According to DSM-IV, she might qualify (barring other diagnoses that trump bipolar disorder, such as a mood disorder due to a general medical condition). But because she does not endorse increased energy phases, she does not quite reach a diagnosis of bipolar II according to DSM-5. Will this tighter requirement increase diagnostic accuracy?

Intuitively, the answer seems obvious: surely the harder it is for a patient to meet all the diagnostic requirements, the harder it should be to award a diagnosis of bipolar disorder. Yet surprisingly, if we look at the statistical determinants of diagnostic accuracy, tighter criteria will have little impact. The reward for working through the following review of these statistical determinants is the understanding that the greatest impact of diagnostic accuracy comes from clinicians, not from tightening criteria.

Diagnostic accuracy: positive predictive value

When a clinician tells a patient “you have bipolar disorder,” how likely is it that the diagnosis is correct? In a widely cited study of overdiagnosis, Zimmerman and colleagues³ found that the answer is “less than 50% of the time.” My essay “Overdiagnosis: Examine the Assumptions, Anticipate New Bipolar Criteria”⁴ questions the broad assumptions that underlie that conclusion. Zimmerman and colleagues derived the 50% diagnostic accuracy figure by calculating a positive predictive value (PPV): in patients said to have bipolar disorder by local clinicians, how many will be found positive by the gold standard (in this case, a structured clinical interview)? In their study, a structured interview did not support the diagnosis in 56% of patients who had been told they had bipolar disorder.

Two key factors determine PPV, and thus determine why in this study the PPV was so low (only 46%). First is the specificity of the test in question, which in this case is the DSM criteria set as applied by the local clinicians. The other determinant of PPV is prevalence-how likely is the illness before the clinician starts trying to diagnose it?

Another related concept needs to be introduced to show that clinicians have more impact on accuracy than the DSM criteria, namely: prevalence is but one form of prior probability. Clinicians can increase prior probability, ie, the likelihood that the patient has bipolar disorder, by assessing other markers of bipolar disorder that are not in DSM.

Increasing diagnostic accuracy

Compare tightening diagnostic criteria (eg, adding an increased-energy criterion to the required symptoms of hypomania to increase specificity) with increasing prior probability (eg, by asking the patient for additional information not in the DSM criteria: family history, age of onset, course of illness, and response to treatment). As a starting point for this comparison, Table 1 (first row) presents the results from the Zimmerman study, with the very low PPV of 46%.

In the second row, using the same starting point, prevalence is held constant while specificity is increased. This reflects the intent of tightening changes in DSM-5 to make it harder to confirm the diagnosis. Although the increase in specificity is small, from 86% to 90%, this is likely in the range of what tightening criteria could produce. Note that with this change in specificity, PPV increases from 46% to 55%. (Greater increases in specificity are clinically very difficult to achieve, but even if specificity were somehow to be increased to 95%, the conclusions presented here still hold.⁵)

Table 1

Increasing positive predictive value (PPV) increases diagnostic accuracy of bipolar disorder

In the third row, specificity is held constant at the same 86% value found in the Zimmerman study, while prior probability is increased to 30%. This is what happens when a clinician has a hunch (eg, by inquiring about family history and response to prior treatment attempts) that the likelihood of bipolar disorder underlying Ms Alvarez’s depression is higher than population prevalence (eg, about 1 chance in 3). Note that now PPV increases to 68%: a better than two-thirds chance of accuracy, up from less than 50% as found by Zimmerman and colleagues, and well above the increase associated with a moderate improvement in specificity shown in the second row.

Table 2

Markers of bipolar disorder not found in DSM-5

Finally, the fourth row shows what happens when a clinician narrows the possibility that the patient has bipolar disorder to a 50/50 chance, before applying DSM criteria. For example, imagine that Ms Alvarez, presenting with depression, had multiple features of bipolar disorder: a first-degree relative with a firm diagnosis; onset of her first depression at age 18; postpartum depression; multiple, brief episodes; a strong adverse reaction to an antidepressant; and no sustained benefit in 4 well-conducted antidepressant trials. With all this, the likelihood that her depression is a bipolar depression is at least 50% (a conservative estimate⁶). With a 50/50 prior probability, use of DSM criteria with a specificity of 86% achieves a PPV of 83%. This is much higher than can be achieved by tightening DSM criteria to a specificity of 90%, or even 95%.⁵

The analysis above suggests that the tightening of DSM criteria will have only a small effect on diagnostic accuracy. A far greater effect can be achieved if clinicians adopt a practice of routinely assessing the non-DSM markers of bipolar disorder shown in Table 2, all of which have been repeatedly associated with bipolar disorder.^6-8 Their presence does not make the diagnosis. Rather, their presence increases the probability that a depression is a bipolar depression.

What good clinicians already do

This is not a novel approach. Routine assessment of these bipolar markers has been part of the assessment of patients at the Harvard-associated mood disorder clinic for 10 years, as part of a Bipolarity Index.⁹ Likewise, Ketter and Citrome¹⁰ suggested that routine assessment of these bipolar markers is “what good clinicians already do.”

Unfortunately, more than half of all mental health care in the US is delivered in primary care settings, where assessment of non-DSM bipolar markers is not “what good clinicians already do.” How can psychiatrists and psychologists improve diagnostic accuracy there? At minimum, they should help primary care colleagues screen as effectively as possible for bipolar disorder.

Screening questionnaires are far from ideal. Their results are too often interpreted as a diagnosis. But they can be used to identify patients who should not receive an antidepressant before further assessment by a mental health specialist. Because the Bipolar Spectrum Diagnostic Scale (BSDS) has been found to be more sensitive for bipolar II, it is preferable for use in primary care, where bipolar II is more likely to be missed than bipolar I.¹¹ The BSDS has been appended with a 1-page questionnaire that assesses all of the factors shown in Table 2.¹² This version of the BSDS does not require further validation, because the additional questions merely gather data for interpretation by a clinician.

Good clinicians will not only routinely assess non-DSM bipolar markers, they will also encourage their primary care colleagues to do the same, using a screening tool developed for this purpose. These steps will do more for improving diagnostic accuracy across the entire mental health care delivery system than using the new DSM criteria.

References:

1. American Psychiatric Association. DSM-5 Development. http://www.dsm5.org. Accessed March 11, 2013.

2. Mitchell PB. Bipolar disorder: the shift to overdiagnosis. Can J Psychiatry. 2012;57:659-665.

3. Zimmerman M, Ruggero CJ, Chelminski I, Young D. Is bipolar disorder overdiagnosed? J Clin Psychiatry. 2008;69:935-940.

4. Phelps J. Overdiagnosis: Examine the Assumptions, Anticipate New Bipolar Criteria. Psychiatr Times. 2013;30(3):12.

5. Phelps J, Ghaemi SN. The mistaken claim of bipolar ‘overdiagnosis’: solving the false positives problem for DSM-5/ICD-11. Acta Psychiatr Scand. 2012;126:395-401.

6. Mitchell PB, Frankland A, Hadzi-Pavlovic D, et al. Comparison of depressive episodes in bipolar disorder and in major depressive disorder within bipolar disorder pedigrees. Br J Psychiatry. 2011;199:303-309.

7. Ghaemi SN, Ko JY, Goodwin FK. “Cade’s disease” and beyond: misdiagnosis, antidepressant use, and a proposed definition for bipolar spectrum disorder. Can J Psychiatry. 2002;47:125-134.

8. Mitchell PB, Goodwin GM, Johnson GF, Hirschfeld RM. Diagnostic guidelines for bipolar depression: a probabilistic approach. Bipolar Disord. 2008;10(1, pt 2):144-152.

9. The “Bipolarity Index.” http://www.psycheducation.org/depression/STEPBipolarityIndex.htm. Accessed March 11, 2013.

10. Ketter TA, Citrome L. Addressing challenges in bipolar diagnosis: what do good clinicians already do? Acta Psychiatr Scand. 2012;126:393-394.

11. Zaratiegui RM, Vázquez GH, Lorenzo LS, et al. Sensitivity and specificity of the mood disorder questionnaire and the bipolar spectrum diagnostic scale in Argentinean patients with mood disorders. J Affect Disord. 2011;132:445-449.

12. Screening for Bipolarity: MoodCheck. http://www.psycheducation.org/PCP/launch/downloadMoodCheck.htm. Accessed March 11, 2013.