Good Clinical Care Requires Understanding Statistics

There are dogmatists (and many of them) of this variety who think that they can be good mental health professionals by simply applying the truths of, say, Freud (or Prozac) to all. This article, and the 2 that will follow in future issues, are addressed to those who know that they do not know or at least want to know more.

When faced with persons who are mentally ill, we clinicians need to first determine what their problems are and then what kinds of treatments to provide. In both cases, particularly in the matter of treatment, we need to turn somewhere for guidance. How should we treat patients?

We no longer live in the era of Galen and pointing to the opinions of a wise man is insufficient (although many still do this). Many have accepted that we should turn to science, that some kind of empirical research should guide us.

If we accept this view-that science is our guide-then the first question becomes: how are we to understand science?

Science is not simple
This article and those that will follow would be unnecessary if science was simple. I would like to disabuse the reader of any simple notion of science, specifically “positivism.” This view holds that science consists of positive facts, each piled on one after another-each of which represents an absolute truth, or an independent reality-and that our business is simply to discover those truths or realities.

This is simply not the case. Science is much more complex.

For the past century, scientists and philosophers have debated this matter. A partial summary might be as follows: facts cannot be separated from theories; science involves deduction, and not just induction. In this way, no facts are observed without a preceding hypothesis.² Sometimes, the hypothesis is not fully formulated, or even conscious: I may hold a number of assumptions that direct me to look at certain facts. In this sense, philosophers say that facts are “theory-laden.” But between fact and theory, no sharp line can be drawn.

A scientific revolution
The rise of statistics was really a revolution; it was a major change in our thinking about science. Before, even the most enlightened thinkers (such as the French Encyclopedists of the 18th century, and Auguste Comte in the 19th century) saw science as the pro­cess of developing absolutely certain knowledge through refinements of sense-observation. Statistics is based on a different concept-the notion that scientific knowledge, derived from observation using our 5 senses that are aided by technologies, is not absolute. Hence, “the basic idea behind the statistical revolution is that the real things of science are distributions of number, which can then be described by parameters. It is mathematically convenient to embed that concept into probability theory and deal with probability distributions.”³

It is thus not an option to avoid statistics, if one cares about science. And if one understands science correctly, not as a matter of absolute positive knowledge but as a much more complex probabilistic endeavor, then statistics are part and parcel of science.² Some doctors hate statistics; but they claim to support science. They cannot have it both ways.

A benefit to mankind
Statistics developed outside of medicine, in other sciences in which researchers realized that uncertainty and error are part of science. Once the wish for absolute truth was jettisoned, statistics became an essential aspect of all science. If physics involves some uncertainty, medicine involves even more. Human beings are much more uncertain than atoms and electrons.

The practical results of statistics in medicine are undeniable. If nothing else had been achieved but 2 things in the 19th century-the end of bleeding, purging, and leeching as a result of the first use of statistical methods by Pierre Louis⁴ in Paris; and in the 20th century, the proof of cigarette smoking–related lung cancer as a result of A. Bradford Hill’s⁵ epidemiological studies-we would have to admit that medical statistics have delivered humanity from 2 powerful scourges.

Numbers do not stand alone
The history of science shows us that scientific knowledge is not absolute, and that all science involves uncertainty. These truths lead us to a need for statistics. Thus, in learning about statistics, the reader should not expect pure facts; the result of statistical analyses is not unadorned and irre­futable fact; all statistics reflect an act of interpretation, and the result of statistics is more interpretation. This is, in reality, the nature of all science: it is all interpretation of facts, not simply facts by themselves. This statistical reality-the fact that data do not speak for themselves and therefore that positivistic reliance on facts is wrong-is called confounding bias.⁶ As I will discuss in the second article in this series, observation is fallible: we sometimes think we see what is not, in fact, there. This is especially the case in research on human beings. Consider: caffeine causes cancer. Nu­merous studies have shown this. The observation has been made over and over again . . . among those with cancer, coffee use is high compared with those without cancer. Those are the unadorned facts-and they are wrong. Why? Because coffee drinkers also smoke cigarettes more than non–coffee drink­ers. Cigarettes are a confounding fac­­tor in this observation, and our lives are chock-full of such confound­ing factors.

Translation? We cannot believe our eyes. Observation is not enough for science. One must try to observe accurately by removing confounding factors. How? There are 2 ways:

• Experiment: This allows us to control all but 1 factor in the environment. We can therefore know that any changes are the result of the impact of that single factor. This can be done with animals in a laboratory, but human beings cannot be controlled in this way (ethically). Enter the randomized clinical trial (RCT). We employ RCTs in experiments with humans to be able to observe as accurately as is ethically feasible.

• Statistics: Certain methods (eg, regression modeling) have been devised to mathematically correct for the impact of measured confounding factors.⁶

Therefore, we need statistics, generated either through the design of RCTs or through special analyses, so that we can make our observations accurate, and so that we can correctly (and not spuriously) accept or reject our hypotheses.

Science is about hypotheses and hypothesis testing, confirmation and refutation, confounding bias and experimentation, and RCTs and statistical analysis: it is not just about facts. Facts always need to be interpreted. And that is the job of statistics: not to tell us the truth, but to help us get closer to the truth by understanding how to interpret the facts.

Knowing less, doing more
The facts are much more complex than they seem. When we appreciate statistics, we end up “knowing” less than we had, in the sense that we realize that much that passes for knowledge is only one among many in­ter­pretations. At the same time, statistical wisdom is liberating: we are less at the mercy of numbers and more in charge of knowing how to interpret them. We will know less, but at the same time, what we do know will be more valid and more solid. In this way, we become better clinicians by applying accurate knowledge rather than speculation, and by being more clearly aware of where the region of our knowledge ends, and where the realm of our ignorance begins.

References:

1. Silverman WA. Where’s the Evidence? Debates in Modern Medicine. New York: Oxford University Press; 1998.
2. Fulford K, Thornton T, Graham G. Oxford Textbook of Philosophy and Psychiatry. Oxford, UK: Oxford University Press; 2006.