Rating Scales: DSM5 Bites Off Far More Than It Can Chew

The problems in the preparation of DSM5 have arisen from its unhappy combination of excessive ambition and poor execution. A prime example is the totally unrealistic ambition to provide diagnostic rating scales for each section of DSM5. The goal is to help standardize interviewing in order to increase diagnostic reliability. Surely, it would be nice to have clinicians gather the most pertinent information in a consistent and systematic way. But there has been (and will continue to be) an insurmountable problem in execution. Creating a universal set of rating scales is an enormous task-one that is well beyond the skill, resources, and time available to DSM5. The scales suggested for DSM5 are (with very few exceptions) ad hoc, poorly written, inconsistent, and untested. I have discussed them with many people who have considerable experience in scale development, testing, and application. They expressed unanimous surprise at the audacity of the DSM5 Task Force in taking on such a massive project and dismay at the poor quality of the initial results. Two representative comments: “it takes a career to get a rating scale right-you don't do it in a few committee meetings” and “the scales look like they were written on the back of a napkin.”

Some background. During the past 50 years, many hundreds of well-established rating scales have been created to cover just about every aspect of psychopathology. It takes years of painstaking iterative work to develop and test a rating scale. The performance characteristics of each of the candidate items has to be tested. The items then have to be revised, tested again, and revised again, and so on until there is confidence that the scale does indeed measure what it is supposed to measure. The testing must be performed in large and representative samples that include those comparison groups most likely to be confused in the differential diagnosis. The scales must also be tested in primary care settings where so many people are diagnosed and treated with psychotropic medication.  

DSM5 ignores virtually all of the herculean labors that have gone into the development and testing of the already available and well established scales. Most of its scales are new and makeshift. They are meant to be field tested during the next 2 years-but they are far too primitive to be ready for field testing and there is far too little time for the necessary iterative revision process. The scales are so cumbersome and time consuming that they would almost certainly be used rarely, it at all, by clinicians.

The DSM5 group is obviously in way over its head and does not have the psychometric expertise to be in the business of making or testing scales. I think it would be wise for DSM5 to cut its losses now and simply drop the scales project before it wastes more time and resources. This will occasion no loss. The scale suggestions are at a very early stage of development, show no promise, and do not deserve further attention. They will be no more than an unwelcome distraction from the enormous effort required if DSM5 is to ever be a useful document delivered on time.

The projected date for a final draft of DSM5 is just a bit more than 2 years away. The criteria sets are still not close to being usable, field trials are yet to be done, and work has not even begun on the mammoth task of writing the text sections. To date none of the DSM5 product has been done well or done on time. Discretion being the better part of valor, DSM5 should beat a prudent and hasty retreat on the ill conceived rating scales project and devote all its efforts to getting the rest of its house in order.

Should the  DSM5 group be stubbornly determined to continue with this project, at the very least there needs to be a review committee of rating scales experts to try to salvage what is salvageable in what has been done so far. It is also necessary to post what are the next steps in testing and iterative revision. Any future DSM5 scale development needs to be done by people with expertise in this area. All in all, the project seems beyond repair and should be shelved now. DSM5 already has its hands more than full.