Data-Driven Approach May Improve Care

Given that passion, opinion, opportunism and inertia have shaped much of managed care's evolution, there is an increasing need for the systematic gathering and rational application of facts. Outcome evaluations and insights into what facilitates and what impedes efficient and effective care are now avidly sought, not only for improving care delivery and treatment effectiveness but also for regulatory functions and commercial promotion.

Having become increasingly involved in academic as well as clinical aspects of managed care, the department of psychiatry at the University of Medicine and Dentistry of New Jersey-New Jersey Medical School, with support from an unrestricted educational grant from Pfizer Inc., is establishing one of the first managed care behavioral health services research fellowships.

Additionally, we have been evaluating aspects of managed care using a set of data accumulated by the telephone access center. The center serves a behavioral health care provider network sponsored by the department of psychiatry. Although created for administrative purposes, the access center's database provides a novel opportunity to study communication and decision-making in psychiatry and managed care operations.

For our first investigation, we selected one sharply defined aspect of therapeutic

decision-making and reported some of our findings at the 1998 American Psychiatric Association Annual Meeting in Toronto (Ciccone et al., 1998). Surprisingly, few data-driven analyses of managed care decision-making were presented at the meeting. In contrast to psychiatry, other mental health disciplines are more actively investigating various aspects of managed care.

Our new research poster focused on how well the access center, using operational criteria, differentiates between callers considered appropriate to be seen first by psychiatrists from those considered appropriate to be seen first by social workers or psychologists. We found a high rate of agreement between the treatment and interdisciplinary referral decisions made by the clinicians who eventually treated the patients and the access center workers' choice of clinician disciplines for the patients' initial appointments (Table).

Information Flow

Physicians commonly grumble that current implementations of the managed care paradigm severely interfere with their relationship to patients. Doctors also express misgivings about whether the electronic interfaces envisioned for telemedicine can convey essential nonverbal material. Many believe that critical information is lost if a patient is not seen face-to-face.

Because behavioral health care is considered especially vulnerable to factors that impinge on the doctor-patient relationship, it should be a good arena for tracking information that contributes to treatment outcome. Specifically, adverse situations such as having to discuss one's psychiatric problems over the phone with a stranger in order to get help should limit the transmission of crucial clinical data in a measurable way.

Our access center included disadvantageous features of both managed care and telepsychiatry, creating a severe test bed for examining whether useful clinical communication could occur over the telephone or in a managed care setting. Although the supposed disadvantages of telephone interviews and impersonal managed care transactions seem clear intuitively, the actual impact of such conditions on clinical communication has not been quantified.

The provider group associated with our department of psychiatry receives fee-for-service under a managed care contract that covers some 130,000 lives for ambulatory services. The multidisciplinary practice sites in the network are located in five counties. As far as we know, this is the largest such operation under academic auspices. Families eligible for services have subscribed through an employer, and most families live in suburban communities, creating a relatively homogeneous population.

The usual entry point for our care system is a centralized telephone access center operated by University Behavioral Healthcare, which is administratively distinct from the academic department of psychiatry. This access center is staffed by professionally trained, clinically experienced care managers (CMs). Their mission is to arrange an appointment for each caller with a network clinician in as timely, convenient and clinically appropriate manner as possible. To do so requires effective communication to ascertain the problems for which the caller wants help.

In a large room in Piscataway, N.J., each CM sits in a sound-isolated carrel equipped with a networked computer. The computer presents checklists and text fields to help the CM conduct a semi-structured interview. It then lists intake appointments that are available at locations convenient for the caller. On the basis of checklist data, the computer shows which available clinicians have special clinical proficiencies that may be required for treating the caller's issue or disorder.

Measuring

CMs are instructed to refer new patients to psychiatrists when clinically necessary and where pharmacotherapy seems indicated, and to social workers or psychologists for other care.

This policy provided us with an outcome criterion for our communication study. We could assess the accuracy of CMs in predicting the need for psychiatrists. If a patient had two or more sessions with a psychiatrist, usually for evaluation for pharmacotherapy, we considered it as evidence that the initial referral to the psychiatrist had been indicated when the patient first called. Otherwise, we assumed such a referral was not indicated. The database recorded CPT codes for all bills submitted to Green Spring Health Services Inc. (now part of Magellan Behavioral Health), so we could readily measure our outcome criterion.

We were able, therefore, to identify the following outcomes: a referral to a psychiatrist that was indicated by our outcome criterion; a referral to a nonpsychiatrist of someone who was subsequently referred to a psychiatrist; a psychiatric referral that did not result in pharmacotherapy (possibly an unnecessary referral, operationally assuming a highly restrictive role for psychiatrists); and a nonpsychiatrist referral that did not ultimately result in referral to a psychiatrist.

A well-functioning CM in this model should have a high rate of correct M.D. and non-M.D. referrals. However, CMs faced formidable obstacles. Although the CMs had no gatekeeper duties in this particular access center, callers may well have viewed them more as blockers than as facilitators of care. In addition, they were telling a stranger about intimate, often stigmatized, problems during a crisis period, or at least when their concern or discomfort had finally induced them to seek professional help. The CMs had to make their decisions after a brief conversation where neither party could see the other to exchange much of the nonverbal communications that clinicians usually deem so important.

Indicated referrals above chance level are manifestations of clinically useful information having been conveyed to and used by the CMs. We could quantify this, as well as the amount of information subsequently available to the treating clinicians, assuming their decisions to be the gold standard. Thus, we had a way of measuring how much "meaning" was coming across on the phone to the CMs compared with the subsequent face-to-face clinical sessions.

Findings

From billing records, we estimated that treating clinicians considered pharmacotherapy or other psychiatric intervention early in treatment for about 29% of the callers to the access center who came to treatment (Table). CMs seemed to recognize this need 62% of the time. In contrast, of those where a medication session was not indicated, CMs sent only one in seven to psychiatrists. Thus, CMs were picking up enough information from most of the callers to be able to determine which patients were to receive psychiatric drugs.

Using information theory, we were able to set constraints on the amount of potentially applicable information callers conveyed to CMs and what proportion of this information was used for decision-making. Despite numerous barriers to meaningful communication and to clinical functioning, the telephone access center was efficiently acquiring and processing information relevant to choosing between a psychiatrist and a nonpsychiatrist.

Considering the sources of uncertainty impinging on the issue, the accuracy CMs achieved in choice of referral discipline was surprisingly high. They were aiming at a moving target. An "outcome" usually crystallizes after a patient attends one or two evaluation sessions and one or more therapy sessions. By then, indications for drug treatment may have changed, and the patient could feel quite differently about self-disclosure and accepting medication.

Furthermore, our outcome criterion precluded crediting CMs when a referral was indicated by nonpharmacologic considerations. For example, some patients may have been referred to psychiatrists just for diagnosis or because of medical comorbidity. This resulted in underestimating the effectiveness of the access center in the domain that we studied. Indeed, the actual proportion of clinically useful information that callers provided may even have approached 100% of the maximum possible. To determine what missed information, if any, could have further improved CM referral accuracy requires a prospective, quality-controlled study that includes re-interviewing callers in depth shortly after their initial call to the access center.

The efficiency of communication that we observed shows that managed care and telepsychiatry, even in tandem, can permit efficient and clinically meaningful transactions. We were able to follow the trajectory of restricted clinical information from the client through the access center to treatment impact and found that much of what was useful had managed to survive the passage.

A similar methodology can be applied to the operationally meaningful components of other classes of information. Therefore, it may be possible to measure with some precision the practical impact of various contingencies on selected verbal and nonverbal communications that contribute to treatment outcomes.

Systematic collection of the data used for our analysis was a by-product of ongoing managed care operations. We plan to study a much larger set of more clinically oriented data that was gathered in the course of case management. Because no survey or other elaborate data-gathering effort is entailed, relatively little research funding is necessary to conduct meaningful studies. The opportunity to examine information about the treatment of a sample of tens of thousands of patients should enable us to assign quantitative constraints to many issues raised in current debates about both managed care and telepsychiatry.

References:

Reference

1.

Ciccone DS, Pulier ML, Castellano C et al. (1998), Psychiatric referral by managed care interviewers. New Research 593. Presented at the 151st American Psychiatric Association Annual Meeting. June 3; Toronto.