Advice on DSM5 Field Trials

The primary purpose of any DSM field trial is to avoid surprises and unintended consequences. The DSM drafts are always created by very gifted experts-but their research and clinical experience (and the scientific literature they review) has usually been derived from the rarified, highly select atmosphere of university specialty clinics. The gnawing question always is how well the suggestions generated in research settings will play out in the rough and tumble world of everyday clinical practice.

Field trials are inherently fallible-no matter how well funded and carefully performed. We were granted more than $2 million of NIMH funding to conduct 11 DSM-IV field trials. Our results predicted that the DSM-IV drafts (compared with DSM-IIIR or drafts of ICD10) would cause no significant change in the rates of any of the disorders studied. The good news was that the field trials were right 9 times out of 11. The bad news was that 2 completely missed the boat and failed to predict the subsequent huge “epidemics” of ADHD and autism.

There are at least 4 reasons why even well-conceived and well-executed field trials can fail so badly:

1. The difficulty in including sites that generalize well to the extremely varied mental health and primary care settings where diagnosis and treatment are done.

2. The unpredictable but possibly powerful influence of drug company marketing (which was important, for example, in swelling the ranks of those with ADHD).

3. The role of expanded school services (eg, important for childhood bipolar disorder and autism).

4. Media fads that can lead to diagnostic fads (eg, Tiger Woods putting hypersexuality on the map).

Of course, field trials that are poorly planned and poorly executed will find any number of other more avoidable ways to fail in the predictive task.

In the past 6 decades, the purpose and methods of field testing have changed. Drafts of DSM-I and DSM-II were given to clinicians in an informal way to determine by survey if they seemed feasible and acceptable. The advance for the DSM-III field trials was the testing of interrater reliability. The DSM-IV field trials introduced the systematic study of how different options affect rates.

At this point in the evolution of field testing, several conclusions can be drawn:

• Trained raters using cleanly written criteria sets can achieve varied (but at least reasonable) reliability.

• The reliability in average clinical practice will be lower-perhaps much lower.

• The field trial–generated rates of any given disorder provide no more than a lower limit for its eventual prevalence in the general population. Actual rates will sometimes be much higher, especially if drug marketing is a factor.

Where does this bring us in regard to DSM5? Unfortunately, the purpose and methods of DSM5 field testing have been shrouded in the usual secrecy, but what has become public reveals great confusion. The original plan was to begin DSM5 field testing last summer without benefit of any public review of the Work Group drafts. Under external pressure, this obviously misguided idea was dropped and field trials had to be postponed for a year-with a planned but unlikely start date of early this summer.

The details have not been posted, but apparently there will be a 2-stage process. In phase 1, proposals will be tested in research and clinical settings. Phase 2 involves clinicians viewing videotaped interviews at professional meetings. We don’t know anything about the goals of the field trial; how the proposals, sites, raters, and subjects will be selected; and how the data collected will answer the questions raised about the potential risk that many of the DSM5 suggestions will cause false-positive epidemics and forensic difficulties.

Obstacles to success

Many obstacles stand in the way of achieving successful DSM5 field trials. Most obviously, the DSM5 drafts are not nearly refined enough to merit field testing. You can’t repair the criteria sets as you test them any more than you can repair an airplane in flight. If the wording of the criteria sets continues to be changed during the testing period, you don’t learn how the final drafts will work. Before field testing makes any sense at all, the very rough DSM5 drafts must be subjected to months of refining through careful public review and painstaking and expert editing.

The next biggest problem derives from the fact that many of the proposed disorders lie at the boundary with normality and are therefore likely to be very common in the general population (eg, binge eating disorder, mixed anxiety-depression, minor neurocognitive disorder, psychosis risk syndrome). There are also many suggested reductions in diagnostic threshold that could result in markedly increased rates for existing disorders (eg, for ADD, major depression, and bipolar disorder). Field testing for these changes can only be done in samples derived from primary care or in reanalyses of data already collected in epidemiological studies-not from the samples of convenience that are likely to be the fallback for the DSM5 trials.

The American Psychiatric Association has so far failed in its efforts to obtain external funding for its DSM5 field testing. To avoid wasting the limited existing in-house resources, it is crucial that the purpose and methods of the DSM5 field trials receive an open and interactive peer review with the field at large. This will be the only way to avoid making continued missteps that lead to repeated blind alleys. All available field trial plans should immediately be posted with an adequate period allotted for public comment and revision. Otherwise, it seems likely that the DSM5 field trials will be a waste of time, money, and effort and will provide no credible support for the changes being suggested.

My advice

1. Devote every available resource to producing clear, consistent, final draft–quality criteria sets before any field testing.

2. For the unfamiliar dimensional proposals, clinician acceptance and reliability remain very uncertain and therefore deserve extensive study.

3. It is well established that reasonably well-written criteria sets achieve reasonable reliability and clinician acceptance. This is no longer sufficient evidence for inclusion in DSM5. Instead, the essential question to be studied is the impact of the suggestion on the rates of mental disorder. This can only be done using primary care sites in field trials or by reanalyzing existing epidemiological data sets.

4. The most important field testing would be of proposals that could create forensic problems.

5. No suggestion identified as possibly having serious unintended consequences should be included without field testing and a careful risk-benefit analysis.