Is Your Signal to Noise Ratio Improving?

Sy Saeed, MD, MS, MACPsych;

Publication

Article

May 9, 2012

Psychiatric Times

Psychiatric Times Vol 29 No 5

Volume29

Issue 5

Is Your Signal to Noise Ratio Improving?

Author(s):

Sy Saeed, MD, MS, MACPsych

With the fast growth of knowledge, the challenge to keep up with the ever-growing body of information is greater than ever.

Psychiatrists and other mental health providers are under increasing pressure to stay current. With the fast growth of knowledge, the challenge to keep up with the ever-growing body of information is greater than ever.

There is an emerging realization that as clinical providers, we need systems or methods to help sort and evaluate the utility of information before we apply it in clinical care. Without such systems, when we are facing an overload of information, most of us tend to take the first or the most easily accessed information-our clinical experience or the drug detailer’s latest data. Perhaps we overestimate the quality and utility of that information while we remain unaware of potentially more useful and relevant information. As a result, the quality of care we provide tends to become suboptimal and the cost of care increases.

Clinical practice is based on knowledge, skills, beliefs, and research. Knowledge can become outdated quickly and skills can get rusty. The traditional methods of improving skills and keeping current with new diagnostic and treatment approaches include reading professional materials, accumulating clinical experience, attending CME conferences, learning from colleagues, and conducting research. There are significant problems, however, with each of these strategies.

The inferences we draw from our individual successes and failures in clinical encounters, with or without evidence, shape our beliefs, which, in turn, influence our clinical practice. These inferences can be distorted by overvaluing treatment successes and undervaluing treatment failures. Evidence-based medicine does not consider one’s clinical experience to be listed among the levels of evidence, and expert opinion is either viewed as the lowest level of evidence¹ or has fallen off the list completely.²

Attending CME events has been shown to have little or no effect on clinical practices.^3,4 Learning from colleagues’ clinical experience can have similar recall problems. Conducting one’s own research is sel-dom done because it is too time-consuming. The challenge is that the knowledge base continues to grow at a rate that is impossible to keep up with unless we devise effective and efficient methods to identify, evaluate, consolidate, and distribute new knowledge and provide tools for real-time analysis and decision support.

Rate of growth in knowledge base

It is estimated that if we took all the knowledge that humankind had accumulated by the year 1 AD as equal to 1 unit of information, it probably took about 1500 years, or until the 16th century, for that amount of knowledge to double. The next doubling of knowledge (from 2 to 4 “knowledge units”) took only 250 years, until about 1750 AD. By 1900, 150 years later, knowledge had doubled again to 8 units.⁵

The rate at which information has been doubling continues to pick up speed. It is now estimated that the collective sum of all printed knowledge is doubling every 4 years.⁶ Between 1999 and 2002, the amount of new information stored on paper, film, magnetic tape, and optical media was estimated to have about doubled, with a growth rate of about 30% a year.⁷ President Clinton⁸ had the following to say about this subject in his 1998 remarksto the National Association of Attorneys General:

The sheer volume of knowledge is doubling every 5 years now. . . . because of human genome research, we are literally solving problems in a matter of days that took years to solve not long before I took office. The World Wide Web is growing by something like 65,000 websites an hour now. When I took office, there were 50-(laughter)-50.

Medical knowledge is now reported to be doubling every 8 years,⁹ and medical professionals are struggling more than they ever have to keep up. Of course, this is not something that is entirely new. Stewart¹⁰ wrote the following almost 50 years ago:

A brilliant student who might successfully master all the present facts and theories by graduation would be seriously out of date 10 years later, and hopelessly so by retirement age unless he continued his education while in practice.

Lifelong learning

Medical education is a lifelong learning process.^11,12 From a consumers’ perspective, our patients expect, and often believe, that their physicians are keeping their knowledge and skills current.

Most medical boards now require that physicians accumulate a significant number of CME hours to re-new their licenses. Medical specialty boards are now also emphasizing maintenance of certification as a way to establish physicians’ lifelong learning. While often we may not think of professional competency as an ethical requirement, ethical background is founded on being primarily responsible for the welfare of the patient through professional competency, striving to continue to learn, being honest with colleagues and patients, and demonstrating responsibility to society.¹³ This concept of professional competency as an ethical responsibility was probably best described by Clifford¹⁴ in The Ethics of Belief: “It is wrong always, everywhere, and for anyone, to believe anything upon insufficient evidence.” Of course, the challenge we face is that knowledge continues to grow, our understanding of illnesses we treat continues to evolve, and our hold on “truth” is never complete.

We know that all scientific truths are provisional! What is concerning to both consumers and professionals is that as research and technology rapidly advance, the gap between what should be done in clinical settings and what is actually practiced appears to be widening. This has led to a paradox: we live in times when the treatment of mental disorders has never been more effective, yet many of our patients do not benefit from these hard-achieved scientific advances.

It is well established that there is a gulf between what we know and what we practice. Large gaps exist between best evidence and practice in the implementation of guidelines.¹⁵ Failure to follow best evi-dence highlights issues of underuse, overuse, and misuse of drugs¹⁶ and has led to widespread interest in the safety of patients.¹⁷ These serious and widespread quality problems have occurred “in small and large communities alike, in all parts of the country, and with approximately equal frequency in managed care and fee-for-service systems of care.”¹⁶

Literature reviews conducted by RAND had previously identified studies that document quality shortcomings. Large gaps between the care patients should have received and the care they did receive were identified. This was true for preventive, acute, and long-term care across all health care settings and for all age-groups and geographic areas.¹⁸ The Institute of Medicine’s major and widely cited reports described this problem as follows: “Between the health care we have and the care we could have lies not just a gap, but a chasm.”¹⁷ Understandably, there is an increasing public concern about the lack of access to appropriate treatment, pervasiveness of unsafe practices, wasteful uses of precious competence, and unsatisfactory patient outcomes.

Now, more than ever, we need novel methods to help us rapidly identify, evaluate, consolidate, and distribute new knowledge. This requires tools for real-time analysis and decision support.

We live in an age of information. But how much of this information is useful, and how much of it gets in the way of learning and keeping up? The answer depends on our ability to find a signal in the noise of this information explosion.¹⁹

The signal to noise ratio [SNR] is a useful concept in determining what information is relevant. The SNR is a qualitative measure of value received relative to the irrelevant data one must sift through to get to that value. In other words, the task at hand is to find the information and knowledge bases that improve medical decision making and can be effectively applied in current clinical practice. Many efforts continue to be made to narrow the gap between evidence and clinical practice. These efforts have included educational strategies to alter practice behavior and interventions at the organizational and administrative levels.^20,21

The Medical Informatics Panel of the Medical School Objectives Project of the Association of American Medical Colleges identified the following 5 major roles played by physicians in the area of medical informatics: lifelong learner, clinician, educator-communicator, researcher, and manager.¹¹

This report further states that to successfully perform as a lifelong learner, a physician must be able to demonstrate competence in many areas, including knowledge of the information resources and tools available to support lifelong learning; ability to retrieve, filter, evaluate, and reconcile information; and attitudes that support the effective use of information technology. While there are many ways to acquire and practice these skills, it is the position of this report that evidence-based medicine provides a coherent approach to do exactly that for patient care.

A “top 25 list” that can help

In future columns, I will illustrate one approach to keeping current with critical findings that are relevant to clinical practice. In an effort to sort and evaluate published research that is ready for clinical use (ie, finding a signal in the noise), I used the following 3-step method to find new information in the previous year that had applicability to clinical decision making and to make a personal judgment as to which information was the most useful:

1. Search select literature published between June 1, 2010, and May 31, 2011.

2. Survey the American Association of Chairs of Departments of Psychiatry, the American Association of Directors of Psychiatric Residency Training, the American Association of Community Psychiatrists, the American Association of Psychiatric Administrators, the North Carolina Psychiatric Association, the Group for Advancement of Psychiatry, and other colleagues with the following question: Among the papers published from June 1, 2010, to May 31, 2011, which ones in your opinion have affected or changed the clinical practice of psychiatry?

3. Look for appraisals in online post-publication reviews, including Faculty of 1000,²² NTK Institute,²³ MDLinx Psychiatry,²⁴ and Evidence Updates From the BMJ Evidence Centre,²⁵ and in secondary sources, such as Evidence-Based Mental Health.²⁶

The papers were chosen on the basis of their clinical relevance/applicability-ie, their “clinic readiness.” The order in which the papers appear in the list is arbitrary.

We need to come up with systems with an acceptable SNR to efficiently identify, evaluate, and apply new information. A method that has been used to develop an annual “top 10” list of important, clinically relevant findings that have been presented at regional psychiatric association meetings and that have been well received has been described. While the method relies on evidence-based thinking, there is subjective judgment involved in selecting the “top” of any list, and this is not an argument that the method or the selection is “best.” The list is useful, nonetheless, and it is hoped that it will be improved by sharing.

In my next column, I will present an expanded, “top 25” list.

References:

References

University of Oxford. Oxford Centre for Evidence-Based Medicine-Levels of Evidence (March 2009).

http://www.cebm.net/index.aspx?o=1025

. Accessed March 12, 2012.

University of Oxford. CEBM (Centre for Evidence-Based Medicine) Levels of Evidence.

http://www.cebm.net/index.aspx?o=5653

. Accessed March 12, 2012.

Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance. A systematic review of the effect of continuing medical education strategies.

JAMA.

1995;274:700-705.

Sibley JC, Sackett DL, Neufeld V, et al. A randomized trial of continuing medical education.

N Engl J Med.

1982;306:511-515.

The Evolutionary Potential of Quakerism. Knowledge doubling.

http://epoq.wikia.com/wiki/Knowledge_doubling

. Accessed March 12, 2012.

Oppenheimer T. Reality bytes: we listen in on the New-Media Moguls-and they’re nervous.

Columbia Journalism Rev.

1996;35(3):40.

How Much Information 2003? School of Information Management and Systems, University of California, Berkeley. October 27, 2003.

http://www2.sims.berkeley.edu/research/projects/how-much-info-2003

. Accessed December 9, 2011.

Clinton WJ. Remarks by the President to the National Association of Attorneys General. March 12, 1998.

http://www.presidency.ucsb.edu/ws/index.php?pid=55611#axzz1g4EYg2zo

. Accessed December 9, 2011.

Carroll J. 10 Major Health Care/Pharmaceutical Trends.

http://www.jimcarroll.com/weblog/archives/000757.html

. Accessed February 28, 2008.

10.

Stewart CB. The adequacy of education for general practice: a medical educator’s viewpoint.

Can Med Assoc J.

1963;88:712-716.

11.

Contemporary issues in medicine-medical informatics and population health: report II of the Medical School Objectives Project.

Acad Med.

1999;74:130-141.

12.

Gonnella JS, Callahan CA, Louis DZ, et al. Medical education and health services research: the linkage [published correction appears in

Med Teach.

2004;26:282].

Med Teach.

2004;26:7-11.

13.

American Psychiatric Association.

The Principles of Medical Ethics With Annotations Especially Applicable to Psychiatry.

Washington, DC: American Psychiatric Association; 1985.

14.

Clifford WK.

The Ethics of Belief

. 1879.

http://ajburger.homestead.com/files/book.htm

. Accessed December 9, 2011.

15.

Eisenberg MJ, Califf RM, Cohen EA, et al. Use of evidence-based medical therapy in patients undergoing percutaneous coronary revascularization in the United States, Europe, and Canada. Coronary Angioplasty Versus Excisional Atherectomy Trial (CAVEAT-I) and Canadian Coronary Atherectomy Trial (CCAT) investigators.

Am J Cardiol.

1997;79:867-872.

16.

Chassin MR, Galvin RW. The urgent need to improve health care quality. Institute of Medicine National Roundtable on Health Care Quality.

JAMA

. 1998;280:1000-1005.

17.

Institute of Medicine.

Crossing the Quality Chasm: A New Health System for the 21st Century.

Washington, DC: National Academies Press, 2001.

18.

Schuster MA, McGlynn EA, Brook RH. How good is the quality of health care in the United States?

Milbank Q

. 1998;76:517-563, 509.

19.

Association of College & Research Libraries. A Progress Report on Information Literacy: An Update on the American Library Association Presidential Committee on Information Literacy: Final Report. March 1998.

http://www.ala.org/acrl/publications/whitepapers/progressreport

. Accessed December 12, 2011.

20.

Haynes RB, Davis DA, McKibbon A, Tugwell P. A critical appraisal of the efficacy of continuing medical education.

JAMA.

1984;251:61-64.

21.

Davis D, Evans M, Jadad A, et al. The case for knowledge translation: shortening the journey from evidence to effect.

BMJ

. 2003;327:33-35.

22.

Faculty of 1000. Post-Publication Peer Review.