Chronic Fatigue Syndrome--A Lesson in Big Science

The articles--15 in all--outlined a growing need in molecular neurosciences for large research projects to solve even larger, seemingly intractable scientific problems. That is quite a contrast from previous years in which single-gene/single-disease problems could be solved in 1 laboratory, involving fewer than 10 (and often fewer than 5) original researchers. This was possible only because scientists were addressing the lowest hanging molecular fruit, studying straightforward genetic problems with well-defined diagnostic criteria and family trees. The larger genetic problems, many of which involve mental health issues, have proved to need a much larger effort to address properly, and are much harder scientific nuts to crack.

In this column I briefly trace the history of research that led to these CFS papers. I will begin with the firing of a director of the CDC, move to a large program funded by the organization, and then end with controversy surrounding the conclusions of this research. What I really want to show is how some of the tallest giants in the scientific landscape are being assaulted. As you shall see, the potential for both great accomplishment and great failure exist when one attempts to use large research studies to address even larger research questions.

THE DISEASE

CFS is defined as an overwhelmingly debilitating fatigue that lasts more than 6 months. Symptoms include memory problems and muscle pain. The vast majority of persons with CFS are female and it afflicts about 1 million persons in the United States each year.

The clinical profile of CFS--even the presence of the disorder itself--was initially met with a healthy dose of skepticism. One of the biggest problems was the inability to define diagnostic criteria rigorous enough to make sense to a test tube. This skepticism may have directly contributed to the firing of a CDC division director. The reason? He diverted almost $13 million allocated by Congress for work on CFS to more promising research on other infectious diseases. In reaction to the transgression, the CDC agreed to restore the funding, using it to launch a major new research initiative. This bureaucratic decision eventually led to the publication of the articles mentioned previously.

The experimental design took 2 unusual steps, 1 at the intake phase, the other at the analysis phase (Figure). Both steps were deployed to do a statistical end-run around more typical intake and analysis approaches. Traditional methods usually involve enrolling patients who believe they have a disease, then using the same researchers who recruited the patients to also analyze data. This traditional intake always creates potentially confounding selection variables: one must choose from a group of patients who are seeking treatment, and CFS patients are desperate for help.

The intake step for this research involved a survey. Researchers randomly polled about 25% of the population of Wichita, Kan, by phone, asking respondents if they were suffering from severe fatigue. Although this strategy was burdened with its own potential confounders, it allowed the researchers to gather a few thousand possible CFS candidates, whether they were seeking treatment or not.

The candidates then went through additional screening tests at a research facility nearby. The idea was to select only those candidates who fit a rigorous diagnostic criterion either for CFS or CFS-related illnesses. These efforts whittled the list down to 172 candidates (58 with CFS, the rest with CFS-like presentations).

Not surprisingly, most patients were middle-aged women. The candidates underwent an additional and quite expensive screening that consisted of 48 hours of testing and burned through almost $2 million of the research funds. These examinations included blood tests (for biochemical and gene expression investigations), cognitive tests, and sleep studies. This initial flurry of activities occurred, in part, as an attempt to address the scientific community's major criticism of the diagnosis of CFS--an inability to create a reproducible clinical profile of the disease.

The second unusual design feature of the experiment was deployed after the data were gathered from the patients; it is to this feature that I turn next.

THE DATA CRUNCHING BEGINS

As you might expect, a massive pyramid of data was generated from the intake work. But the executive leadership of the experiment did not allow the researchers who collected the data to analyze it.

Instead, quadruplicate sets were made and the data were given to 4 separate "analysis teams." These teams were composed of statisticians, epidemiologists, clinicians, and even physicists who were given very specific tasks. One team, for example, was asked to look for statistical patterns in patients related to their overall health. Height and weight, presentation of depression, sleep disturbances, and so on were examined. Another team was asked to look at CFS directly, but in association with genetic mutations known to be involved in stress responses. They were tasked with looking at mutations in 11 genes associated with the hypothalamic-pituitary-adrenal axis, part of the stress response in which glucocorticoids such as cortisol are deeply involved. A total of 43 mutations scattered across these 11 sequences were reviewed.

The 4 teams were then allowed to crunch through their statistical assignments, a process that took more than 6 months, and the 14 papers were generated from the bulk of their work. Among their most important results were a general and a specific genetic finding.

General finding

A strong biological basis for CFS appeared to exist. The patterns of more than 24 genes were expressed differently in CFS patients compared with controls. The genes varied wildly in function, from sequences involved in potentiating the immune system to the way cells signal each other at the molecular level.

Specific finding

A specific biological basis for CFS appeared to exist as well. Mutations in 3 genes correlated strongly with CFS, including a gene encoding a glucocorticoid receptor and a gene encoding a protein involved in regulating serotonin levels.

The unusual design of these investigations--and the amount of money thrown at them--was created to settle the mystery of the biological underpinnings of CFS once and for all. Yet, even given the large amount of data, settling is hardly what happened.

GRUMP FACTOR REVISITED

I like to think I am a nice guy, but I tend to be a fairly grumpy scientist. The problem almost always lies somewhere in the midpoint between the diagnostic criteria employed to study a question and the correct mathematical model that will allow both findings and interpretation of findings to align. The research literature is littered with projects whose experimental design failed to be rigorous enough to allow the correct view to unfold.

That is why, given its unusual nature, this research is so interesting. Yet, almost as soon as these findings were published, they touched off a firestorm of controversy.

One of the first criticisms was that the statistical sample was too small. Although the investigators cast their net broadly (assaying 25% of the population of a major US city is no small feat), they ended up with fewer than 100 patients who met their diagnostic criteria. That sample was too small for many in the research community. True, the data met the threshold requirements of statistical significance--but just barely. To their credit, the researchers acknowledged this limitation and plans are in the works for further study.

The second criticism had to do with the number of mutations studied (43) in the 11 candidate genes. While it is beyond the scope of this column to discuss the subtleties of linkage mathematics, some researchers believe this is not enough to rule out other variants. Many linkages besides those identified could correlate as closely, they assert, and the data have not conclusively ruled out these other sequences.

The third criticism has to do with the functionality of the identified genes. It is one thing to look at a sequence of DNA, it is another to show that the gene is active. The job of a class II gene is to create a functional messenger RNA (mRNA). But there are many class II genes scattered throughout the genome with sequences so mutated they are no longer capable of making a functional mRNA. The phenomenon is so widespread that the genes are even given their own name: pseudogenes. Other genes are fully functional but are rendered "silent" for developmental reasons. A lot of neural genes remain silent in adipocytes, for example, and are so because the tissues are differentiated to consist of fat cells, not neurons.

The way to solve this problem is not by going after the gene sequence, but by going after the mRNA derived from the sequence. That answers the questions: Is the gene capable of becoming active? And, if so, is it active? The expressive functionality of the genes was not addressed in the initial work, hence the criticism.

CONCLUSIONS

What can we safely conclude from such large studies? First, a big salute is in order for the initial researchers, both for the rigor they employed and their novel approaches to finding solutions. CFS is a big problem, one not easily solved, especially when issues of diagnostic criteria meet a condition that probably involves many, many genes. Second, the effort serves as an example of just how many pitfalls await researchers who do this kind of work, even when there are many.

Happily, the results and the reactions to them have not discouraged the CDC from continuing to address these issues or even from moving forward with new projects using the same novel design. My guess is that we will be seeing more and more authors and articles that address big scientific questions. One can only hope they will be as thoughtful and honest as these teams have been when it comes time for analysis.

Dr Medina is a developmental molecular biologist and private consultant, with research interests in the genetics of psychiatric disorders. *