Coming Soon: The Computer-Assisted Diagnostic Interview

The Traditional Diagnostic Interview (TDI) is the basic technique clinicians use to make their initial psychiatric assessment. TDI consists of broad areas (chief complaint, history of present illness, past psychiatric history, family and personal history, mental status, diagnosis and treatment plan), with specific items in each area.

In 1996 I compared medical records from 10 different hospitals and clinics (including four university hospitals) and found that TDI is standardized in format-all hospitals used printed protocols for recording TDI. I also discovered that TDI is not standardized for content, the protocols differed widely from each other. So, too, many studies have shown that TDI often leads to inaccurate diagnoses.

The inaccurate diagnoses arrived at with use of the TDI have three main causes:

1. Incomplete data base: Lipton and Simon found that 80% of medical charts do not contain enough information to justify the stated diagnosis.

2. Incorrect/incomplete recording of data: May and Miller found that the longer you wait to write down your data, the more quantity and accuracy you lose-up to 50% in the first hour, 75% in the first eight hours.

3. Incorrect application and recall of DSM criteria. Skodol and colleagues found that 75% of incorrect diagnoses result from incorrect application of DSM criteria. Robinson and others found that 13% to 48% of the criteria for major depression were misidentified by a university-affiliated faculty. The written exams I give psychiatric residents find that they recall only about 50% of criteria for schizophrenia. They remember most symptoms but forget criteria about social/occupational dysfunction, duration of illness and other diagnostic exclusions.The most widely used statistical measure of diagnostic accuracy is interrater reliability.

Researchers regularly find that TDI has very low reliability. My own studies, in reports to the Los Angeles County Department of Mental Health (1995) and the corresponding Department of Mental Health Services (1997), agree: Using global diagnoses (defined by the first three digits of DSM-IV code, so that 295=all schizophrenia, 296=all mood disorders, 304=all drug dependence, etc.), we studied 50 patients who were diagnosed first in the emergency room and then on the inpatient service and found kappas of 0.01 for schizophrenia and 0.22 for mood disorders. For 100 outpatients who were diagnosed in 15 different clinics, kappas were 0.03 for schizophrenia and 0.16 for mood disorders for each pair of diagnosticians in sequence. When the statistical measure kappa is below 0.40, it indicates poor interrater reliability and inaccurate diagnoses.

The problem of TDI's diagnostic accuracy, as measured by interrater reliability, is almost totally ignored by clinical and academic psychiatry.

1. The two largest textbooks (Hales and colleagues; Kaplan and Sadock) have references about diagnostic reliability concerning methodology, epidemiology and specific areas of psychopathology, but none about reliability of TDI in real-time clinical practice.

2. The American Psychiatric Association bibliography of psychiatry, Core Readings in Psychiatry (Sacks and colleagues) contains 11 references dated 1961 to 1987 on diagnostic reliability, all of them about theory or statistics, none about real-time clinical practice using TDI.

3. A Medline search on interrater reliability for diagnosis (1990 to 1996) found 111 references. Most focused on a specific area of diagnosis (e.g., negative symptoms, bizarre versus nonbizarre delusions, schizophreniform disorder). Only two were about the real-time use of TDI as a global process.

Whenever research (including clinical trials) requires diagnostic reliability, investigators supplement or replace TDI with structured interviews or rating scales. But clinicians continue to use TDI alone, in spite of its proven limitations. Seeking to provide an improved assessment tool, I am now developing the Computer-Assisted Diagnostic Interview (CADI).

Description of CADI

Computer-Assisted Diagnostic Interview (CADI) uses the computer to assist, enhance and improve TDI. CADI was first presented at the APA's annual meeting in 1996. CADI modifies both data collection and data processing.

Data collection is done with a protocol. The use of protocols during TDI is nearly universal. After examining dozens of protocols from many facilities, conferring with colleagues and consulting textbooks, I developed a three-page protocol that uses 250 data items (most protocols use 50 to 150 items). CADI protocol comes in two forms, printed and computerized. The clinician fills in the printed form either during or at the end of a TDI. All protocol items are number-coded so that the clinician or an assistant can enter the data into the computer.

Each page of the protocol is displayed on the computer screen, one item at a time. Sample questions are provided alongside each item to help beginning clinicians avoid leading or close-ended questions. Numeric answers are entered by choosing an integer from multiple choices. Narrative answers are typed and stored as text. The user can jump to other screens and change an answer by overwriting.

If the user is only interested in diagnosis, areas such as family and personal history can be ignored and only the Mental Status and DSM diagnostic criteria would need to be entered.The CADI computer program processes data by comparing all items in the data base with all DSM algorithms for adult disorders and finding all diagnoses. CADI then provides the following output:a) Match: the data base matches with enough DSM criteria to make the diagnoses.

b) Partial match: the incomplete data base matches some DSM criteria, but not enough to rule the diagnosis either in or out. CADI specifies which unevaluated criteria are needed to complete the evaluation.

c) No match: the data base has enough negative matches with DSM criteria to rule out these diagnoses.

CADI's Place in Psychiatric Practice

CADI occupies a place between the less-than-reliable TDI and the reliable but time-consuming structured interview like the Structured Clinical Interview for DSM (SCID).

My UCLA inpatient service is staffed by four residents, three medical students and me. We use CADI to collect, process and analyze our data bases.

My experience is that CADI takes about the same amount of time as the TDI. Collecting a larger data base takes more time, but data can be entered faster by checking off items rather than writing answers; the program makes the diagnosis and prints the write-up, saving time with those tasks.

CADI can have a salutary influence on our cognitive attitudes: it leads us to 1) collect a complete data base instead of stopping when we believe that we have minimally enough information; 2) strive for accuracy in obtaining data during the interview; 3) consider a broad differential diagnosis rather than a narrow single diagnosis.

In addition to clinical and research uses, CADI is also useful for evaluating quality/utilization management. With its parameters for measuring change (current scales built into CADI include the Positive and Negative Syndrome Scale for Schizophrenia, Beck and Hamilton Depression Scales, Childhood Autism Rating Scale and the Abnormal Involuntary Movements Scale), we can assess change over time. For managed care, this means that we can justify admissions and uphold requests for extensions of stay through measurements showing that the patient has not yet improved enough for discharge.

Dr. Miller is associate clinical professor and educational director of inpatient psychiatry at Olive View-UCLA Medical Center.

References:

References

1.

Hales RE, Yudofsky SC, Talbott JA. Textbook of Psychiatry, 2nd ed. Washington: American Psychiatric Association; 1995.

2.

Kaplan HI, Sadock BJ, eds. Comprehensive Textbook of Psychiatry, 6th ed. Baltimore: Williams & Wilkins; 1995.

3.

Lipton AA, Simon FS. Psychiatric diagnosis in a state hospital: Manhattan State revisited. Hosp Community Psychiatry. 1985;36:368-373.

4.

May J, Miller PR. Note-taking and information recall. J Amer Med Educ. 1977;52:524-525.

5.

Robinson EP, Asnis GM, Harkavy Friedman JM. Knowledge of the criteria for major depression: a survey of mental health professionals. J Nerv Ment Dis. 1985;176:480-484.

6.

Sacks MH, Sledge WH, Warren C, eds. Core Readings in Psychiatry. An Annotated Guide to the Literature. 2nd Ed. Washington: American Psychiatric Press; 1995.

7.

Skodol AE, Williams JBW, Spitzer RL, et al. Identifying common errors in the use of DSM-III through supervision. Hosp Community Psychiatry. 1984;35:251-255.