Why Evidence-Based Medicine Cannot Be Applied to Psychiatry

Evidence-based medicine (EBM) is rapidly becoming the norm. It is taught in medical schools and is encouraged by both government agencies and insurance plan providers. Yet, there is little proof that this model can be adapted to fit psychiatry.

EBM supposedly allows the clinician to offer the most effective treatment for each patient.^1,2 This goal is laudable, but the model is not appropriate for psychiatry because precise and stable diagnostic criteria are lacking in our specialty. Treatment outcomes in psychiatry are not defined by remission or cure. Instead, fractional reductions in the number and severity of symptoms are accepted, as measured by rating scale scores. Evidence-based psychiatry (EBP) is an untested hypothesis; for this theory to be either useful or valid, 3 basic assumptions must be examined.

• Is the diagnostic system valid?
• Are the data from clinical trials assessing efficacy and safety sound?
• Are the conclusions in a form that can be applied in clinical practice?

Definitions
The paradigms that define EBM (and EBP) are based on data in published clinical studies. Each study is assessed according to the methods used to collect the data and a value placed (by the reviewer) on its quality. EBM uses 3 types of evidence³:

• Grade A: Randomized clinical trials, homogeneous populations, placebo-controlled.
• Grade B: Randomized clinical trials, heterogeneous populations, not placebo-controlled.
• Grade C: Observational studies, case collections, open clinical trials.

The strength of the evidence decreases from Grade A to Grade C, with more weight given to Grade A than to Grade C studies.

The diagnostic system
The DSM represents diagnostic groupings developed through discussion and consultation. These groupings are not based on experimental evidence. The manner in which the DSM creates diagnoses assumes that psychiatric illnesses can be divided into separate categories and that each illness is unique. The process is circular. It begins by assuming that discrete categories exist and produces a document that divides psychiatric illness into discrete categories.

The illnesses in the DSM are delineated by phenotypic features, with a contribution from the patient's recall of the course of the illness. The separation of classes based on these criteria is, by its nature, imprecise. Although these criteria are intended to separate clinical entities, their descriptions are overlapping. The judgments introduced by the clinician's need to decide which of several conditions best meets the diagnostic criteria are subjective, putting the system in doubt. It is not surprising that overlaps are common.

Major psychiatric disorders overlap
The committee-driven DSM classification divides psychiatric diseases into discrete entities based on cross-sectional symptoms and signs. These disorders exist as syndromes and not as specific illnesses. Psychosis, for example, may result from drug toxicity, neurological illness, trauma, or as a feature of delusional depression and isolated delusional states. The genetic predispositions for schizophrenia and affective illness overlap.^4,5 Kendall⁶ has demonstrated that most patients have characteristics of both groups and that our diagnostic concepts are based on the extremes of what is better visualized as a continuum.

EBP relies on categories that call for specified algorithms for treatment.⁷ However, in clinical practice the emphasis is not on treating syndromes or diseases but on applying empirically derived prescriptions for symptoms and symptom complexes. Psychiatry lacks antischizophrenia, antidepression, or antianxiety disorder medications. What exist are medications that symptomatically treat psychosis, depressed mood, and anxiety. The severity and associations of the symptoms vary greatly depending on genetic traits, environmental influences, and duration. The genetic traits are particularly subtle; the substitution of a single nucleic acid markedly alters symptoms.^8-10 Variations in genetic polymorphism influence phenotypic presentations of illnesses over a spectrum of clinical syndromes. It is difficult to reconcile these observations with the current diagnostic system's separation of these conditions into distinct categories and subcategories.

Several conditions exhibit such high rates of comorbidity that one must be skeptical of the idea that we are dealing with discrete entities. The overlap between attention-deficit disorder (ADD), oppositional defiant disorder, and conduct disorder is an example.^11-13 Separating these syndromes into distinct entities obscures the possibility that they may be expressions of a single genotype. This is also the case with obsessive-compulsive disorder (OCD), generalized anxiety disorder, panic disorder, and major depressive disorder, in which patients may display symptoms of each condition, with the dominant symptom changing with time and circumstance.^14-17

This relationship frequently causes problems when the nosological diagnosis obscures the possibility of a common biological substrate. A common example of this occurs in patients with OCD in whom manic symp- toms develop when they are taking antidepressants.¹⁸

If a nosological approach were reliable, we would expect greater homogeneity in treatment response. That is, if depressive mood disorders were a single entity, we would expect that a single treatment agent would be effective in all or almost all patients who receive this diagnosis. Such uniformity of diagnosis and treatment response is expected, for example, in bacterial infections and in diabetes. However, this is not the case in psychiatric disorders. Lacking effective predictors or tests, the clinician searches for clues for treatment selection in personal and family history or engages in multiple drug trials, augmentation strategies, and polypharmacy.

Syndromes that are separated by DSM criteria as single entities often respond to the same pharmacological therapies. OCD, panic disorder, generalized anxiety disorder, and depressive disorder respond to SSRI agents. Schizophrenia, bipolar disorder, tox-ic psychosis, and major depressive disorder with psychosis respond to atypical antipsychotic agents. Are we to assume that the treatment agents have very broad effects, and that they are effective for different disorders? Or should we assume that the different disorders have a common biological underpinning that responds to the singular effects of specific medications? Either assumption casts doubt on the present diagnostic schema.

Similarly, diagnostic categories are added or deleted in each iteration of the classification based on fashion and political correctness. Examples include the rejection of homosexuality, unipolar mania, and melancholia, and the addition of caffeine and nicotine addiction. When fashion, rather than scientific evidence, dictates diagnoses, the entire system should be questioned.

DSM categories
The DSM categorizes illnesses into discrete entities, using 5 axes to assign severity and plausible cause. Axis I is a description of the presumed clinical disorder. Many conditions share comorbidities of 70% to 90% (eg, ADD, oppositional defiant disorder, conduct disorder, and antisocial personality disorder), thus blurring diagnostic boundaries. Conditions change from one form to another depending on when the patient is seen (eg, OCD, generalized anxiety disorder, panic disorder, atypical affective disorder).

Because physicians are limited to a few tests to define a disorder (serology for syphilis, blood glucose for diabetes, and neuroimaging for dementias come to mind), the diagnosis takes on the hallmarks of a Rorschach projective test, based on the judgment and experience of the observer.

Conversely, when psychiatrists use a system based on a checklist of symptoms, disorders that are etiologically related are viewed as different. Tucker¹⁹ gives the example of Huntington disease, a condition in which the cause is well defined but the expressions of the illness meet the criteria for dementia, psychosis, depression, and antisocial personality disorder.

Data validity
EBM places heavy emphasis on the results of large, multisite, double-blind studies. This being the case, it is important to know where and how these studies are conducted.

For the most part, studies are commissioned by industry consultants and are conducted either at institutions that do clinical trials for profit or at academic institutions where such studies are considered of low merit and the actual data collection is done by nurses, social workers, or narrowly trained raters with no clinical experience. Participants are often enrolled from databases of patients who have participated in previous studies or who are "borrowed" from clinics. It is not uncommon for patients to participate in multiple studies. Participants become "professional" research patients who take part in several studies to collect the stipend. These populations are not representative of the patients in clinical psychiatric practices.

Assuming the best of circumstances--that is, the participants represent a cross-section of patients with a defined psychiatric illness--the validity of double-blind, placebo-controlled evidence must still be assessed. Ideally, neither the patient nor the investigator should be able to distinguish the active substance from placebo; however, numerous experiments show that the "blind" condition is easily broken.

In studies of antidepressants, physicians correctly distinguished the active drug in 73% to 89% of cases, and patients correctly distinguished the treatments 64% to 75% of the time. In crossover studies, physicians successfully identified the active drug 100% of the time and patients were correct 93% of the time.²⁰

The failure to report negative findings is most egregious in assessing efficacy. In addition to the reluctance of editors of journals to publish negative findings, industry project managers "seal" negative results and neither file the data with the FDA nor allow the investigators to report the results.²¹ For every study of a psychotropic drug that reports positive results, there are often 4 to 8 unpublished studies that fail to show superiority to placebo.²² This "file-drawer phenomenon" explains, in part, why drug company-supported research is 4 times as likely to produce a positive response as studies from other sources.²³ If the results of the studies that did not show positive results were reported, it is conceivable that they would alter the conclusions of the meta-analysis.⁷

Newer agents that may be patented are preferentially studied, diluting evidence that supports the effectiveness of older treatments and encouraging the wider use of newer and more expensive treatments.²⁴Dosage problem
Clinical trials underestimate the effective dose of medications; they may also overestimate, but this is unusual. The "evidence" is then used by insurance companies and government agencies to accept or reject claims for payment for the higher doses often needed in clinical practice, depriving patients of adequate treatment. The Table illustrates the differences between the originally suggested doses of atypical antipsychotics and what are now accepted as the actual clinical doses.²⁵Application to clinical practice
It takes considerable time for a clinical observation to be validated by a large-scale study.^2,26 In practice, a medication is first approved for marketing as an effective treatment for one entity and then clinical observations reveal that it has other uses. If we assume that it will generally take up to 4 years for a clinical trial to be designed and executed, EBM will always be several years behind clinical practice. If the standard of clinical practice is limited to evidence-based studies, practitioners will be reluctant to attempt novel treatments or to vary dosing regimens. The evaluation process reduces innovation.

EBM is favored by insurance companies and national health services.^27,28 This method is, by its very nature, conservative, and it promotes the use of lower doses of medication and discourages polypharmacy. Such restrictions and standards reduce costs, but they also reduce effective treatments. The problem is compounded when we consider that EBM criteria are commonly offered as the basis for malpractice litigation.

Conclusion
Critics of present practices call for narrow diagnostic criteria in patient selection, better monitoring of the patients selected for studies, longer periods of study, and better criteria for outcome. Some relatively simple measures that would be helpful are monitoring the participants by their social security numbers (to minimize the use of "professional patients"), tracking placebo response percentages, and ensuring better reporting of adverse events. The common practice of competitive enrollment, which encourages research sites to enroll as many patients as possible, should be discouraged because it magnifies the contributions made by specific research installations and skews results even further.

EBP is a system that is based on unreliable data. Its unreliability results from an imprecise and poorly founded diagnostic system, inaccurate data collection, and obfuscation of experience by the pharmaceutical industry and compliant academic leaders. Whatever the reasons, the drive to adopt EBP as a standard of practice is best discouraged.

 

References:

References

1.

Sackett DL, Rosenberg WM, Gray JA, et al. Evidence-based medicine: what it is and what it isn't.

BMJ.

1996;312:71-72.

2.

Sackett DL, Strauss SE, Richardson WS, et al.

Evidence-Based Medicine: How to

Practice and Teach EBM

. 2nd ed. New York: Churchill Livingstone; 2000.

3.

Gray GE. The philosophy and methods of evidence-based medicine: an introduction for psychiatrists.

Dir Psychiatry

. 2002;22:165-173.

4.

Maier W, Lichtermann D, Minges J, et al. Continuity and discontinuity of affective disorders and schizophrenia. Results of a controlled family study.

Arch Gen Psychiatry.

1993;50:871-883.

5.

Rende R, Hodgins S, Palmour R, et al. Familial overlap between bipolar disorder and psychotic symptoms in a Canadian cohort.

Can J Psychiatry

. 2004;50:189-194.

6.

Kendall RE. The major functional psyches: are they independent entities or part of a continuum? In: A Kerr, H McClellan, eds.

Concepts of Mental Disorder

. London: Gaskell; 1991:1-16.

7.

Williams DD, Garner J. The case against "the evidence": a different perspective on evidence-based medicine.

Br J Psychiatry.

2002;180:8-12.

8.

Weinberger DR, Egan MF, Bertolino A, et al. Prefrontal neurons and the genetics of schizophrenia.

Biol Psychiatry.

2001;50:825-844.

9.

Goldberg TE, Egan MF, Gschidle T, et al. Executive subprocesses in working memory: relationship to catechol-

O

-methyltransferase Val158Met genotype and schizophrenia.

Arch Gen Psychiatry.

2003;60:889-896.

10.

Numakawa T, Yagasaki Y, Ishimoto T, et al. Evidence of novel neuronal functions of dysbindin, a susceptibility gene for schizophrenia.

Hum Mol Genet (England).

2004;13:2699-2708.

11.

Burt SA, Krueger RF, McGue M, Iacono W. Parent-child conflict and the comorbidity among childhood externalizing disorders.

Arch Gen Psychiatry.

2003; 60:505-513.

12.

Maughan B, Rowe R, Messer J, et al. Conduct disorder and oppositional defiant disorder in a national sample: developmental epidemiology.

J Child Psychol Psychiatry.

2004;45:609-621.

13.

Thapar A, Harrington R, McGuffin P. Examining the comorbidity of ADHD-related behaviours and conduct problems using a twin study design.

Br J Psychiatry.

2001;179:224-229.

14.

Cassano GB, Pini S, Saettoni M, Dell'Osso L. Multiple anxiety disorder comorbidity in patients with mood spectrum disorders with psychotic features.

Am J Psychiatry.

1999;156:474-476.

15.

Katerndahl DA, Realini JP. Comorbid psychiatric disorders in subjects with panic attacks.

J Nerv Ment Dis.

1997;185:669-674.

16.

Richter MA, Summerfeldt LJ, Antony MM, Swinson RP. Obsessive-compulsive spectrum conditions in obsessive-compulsive disorder and other anxiety disorders.

Depress Anxiety.

2003;18:118-127.

17.

Sanderson WC, DiNardo PA, Rapee RM, et al. Syndrome comorbidity in patients diagnosed with a

DSM-III-R

anxiety disorder.

J Abnorm Psychol.

1990; 99:308-312.

18.

Rihmer Z, Barsi J, Belso N, et al. Antidepressant induced hypomania in obsessive-compulsive disorder.

Int Clin Psychopharmacol

. 1996;11:203-205.

19.

Tucker GJ. Limitations of the

DSM-IV

as a diagnostic tool.

Dir Psychiatry

. 2000;20:127-135.

20.

Shapiro AK, Shapiro E.

The Powerful Placebo.

Baltimore: Johns Hopkins University Press; 1997:190-201.

21.

Vedamtan S. Drugmakers prefer silence on test data.

Washington Post

. July 6, 2004:A01. Available at:

http://www.washingtonpost.com/wpdyn/articles/A29576-2004Jul5.html

. Accessed September 21, 2007.

22.

Sussman N. Misuse of evidence: an open secret.

Prim Psychiatry

. 2003;10:14.

23.

Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review.

BMJ.

2003; 326:1167-1170.

24.

Bauer MS. How solid is the evidence for the efficacy of mood stabilizers in bipolar disorder?

Dir Psychiatry.

2005;25:165-181.

25.

Weinberger D, Stahl SM.

Genes, Circuits, and Pharmacology: Clinical Connections in Schizophrenia

. Archived satellite presentation. Carlsbad, Calif. NeuroScience Education Institute; July 2004.

26.

Egger M, Smith GD, O'Rourke K. Rationale, potentials, and promise of systematic reviews. In: Egger M, Smith GD, Altman DG, eds.

Systematic Reviews in Health Care: Meta-Analysis in Context

. London: BMJ Books; 2001:3-19.

27.

Baker M, Kleijnen J. The drive towards evidence-based health care. In: Rowland N, Gross S, eds.

Evidence-Based Counseling and Psychological Therapies: Research and Applications

. Philadelphia: Routledge; 2000:3013-3029.

28.

Reynolds S. The anatomy of evidence-based practice: principles and methods. In: Trinder L, Reynolds S, eds.

Evidence Based Practice: A Critical Appraisal.

Malden, Mass: Blackwell Science; 2000.