The limits of evidence-based medicine

2008 August 17
more by

Podcast: Download

By David Annis
Article ID: 1244

For many years, doctors have been free to practice medicine in almost any way that they saw fit.  They could use drugs to treat conditions for which they were not approved and use any procedure on any patient from whom they obtained informed consent.  At times this led to doctors treating patients in a manner not supported by scientific research.

Recently, there has been a vigorous movement to get doctors to practice “evidence-based medicine”.  This movement advocates treating every disease based on the available scientific evidence about treatment efficacy and risk.  I have been a strong proponent of this; after all, how can I oppose those who espouse belief based merely on faith and then not support evidence-based medicine?

However, I’ve lately been forced to rethink my position on evidence-based medicine.  It sounds good initially, but there are limitations to its real-world applications:

  1. The subjects in drug trials and other studies are seldom an exact match for the patients that a doctor is treating.  The trial may have been done using patients in a different age range and physical condition.  Most trials purposely exclude people with medical issues other than the one being studied. A treatment that was well tolerated in a patient population with a median age of 35 may not be appropriate for a 60 year old with renal insufficiency.
  2. Diagnostic categories are sometimes not exact.  For example, a patient may have some symptoms that are consistent with ADD and others that are consistent with Autistic Spectrum Disorder.  A doctor – forced by the insurance company to choose – may diagnose ADD, but still wish to treat for symptoms of Autism Spectrum Disorder.
  3. All evidence is not equally good.  Some studies are better designed than others, and sample sizes and levels of statistical significance vary.  There have been attempts to assign a level to each piece of evidence to indicate its quality, but there are still differences within a level. And levels can sometimes be misleading based on the rules used to classify the evidence.
  4. Even if the quality of the evidence is classified properly, evidence can be contradictory.  How many weaker studies are needed to overcome the results of a stronger study? The answer is subjective.
  5. The process of developing a set of evidence-based guidelines is not an entirely impartial and objective process. Insurance companies try to influence the process in a way that keeps costs low. Drug companies try to influence the process in a way that ensures their products are included in the standard protocol.  Those that evaluate the scientific evidence may have their own biases affecting the outcome.

A good doctor looks at the specific individual in front of him, uses his knowledge of how the body works, the patient’s specific constellation of diseases, the patient’s physical and mental state, and decides on the course of treatment that is best for that individual.  Attempting to reduce that clinical judgment to a flow chart and checklist does a disservice to both the patient and the physician.

I am certainly not advocating allowing doctors to treat pain with voodoo dolls and crystal power, but I do believe that we are in danger of allowing the pendulum to swing too far.  Physicians are highly educated professionals, dealing with highly complex, poorly understood systems, including the human mind and body, disease causing organisms, and the natural fauna that exists in and on all of us. They also have to process a continual stream of new information and new discoveries.  Reducing clinical care to a set of checklists and decision trees is a disservice to us all.


Did you like this article? Want more? Support Digital Bits Skeptic.

Other articles related to this topic:

from → Healthcare, Pop Culture

2 Comments
2008 September 14
Steven Novella permalink

David,

I think you need to read a bit more about Evidence-Based-Medicine (EBM). All of the concerns you raise are taken into consideration when evaluating the evidence for an intervention. Reviews are generally done by academics, many specifically done for Cochrane or other EBM databases – insurance companies and pharmaceutical companies have no input. The biases of the reviewers is also a factor, of course. And the literature is a moving target. So seeing multiple reviews is helpful. Also – groups like Cochrane have very strict published methods for how to do a systematic review and score a treatment. This does not remove judgment from the equation, but does try to be as consistent as possible.

Further – EBM is NOT about reducing the practice of medicine to a flow chart. This is really just a misconception. EBM is about reviewing evidence and putting it into the hands of clinicians at the point of patient care. It recognizes explicitly the role of the experience and judgment of a clinician and the individualization of care. Really, you completely mischaracterized EBM in your last paragraph.

From the Cochrane website: http://www.cochrane.org/docs/ebm.htm

Evidence-based health care is the conscientious use of current best evidence in making decisions about the care of individual patients or the delivery of health services. Current best evidence is up-to-date information from relevant, valid research about the effects of different forms of health care, the potential for harm from exposure to particular agents, the accuracy of diagnostic tests, and the predictive power of prognostic factors [1]. Evidence-based clinical practice is an approach to decision-making in which the clinician uses the best evidence available, in consultation with the patient, to decide upon the option which suits that patient best [2]. Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence-based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research [3].

2008 September 18
David Annis permalink

I am not trying to say that there is no value in EBM, that there are not attempts to be as consistent as possible, or that in best practice the clinician’s judgment is not valued. However, there are legitimate concerns surrounding all of these areas that affect EBM’s implementation in the real world. I have read about and spoken to physicians that feel they have been constrained based on EBM.

These concerns are not unique to me. See, for example:
http://medir.ohsu.edu/~hersh/ebcm-04-ebm.pdf
http://content.onlinejacc.org/cgi/content/full/4/5/954?ijkey=f102d3012381e501834c80f23bbcb89de29ae33c&keytype2=tf_ipsecsha

A letter to the editor of Anesthia Analgesia ( http://www.anesthesia-analgesia.org/cgi/content/full/95/6/1817 ) begins “Proponents of evidence-based medicine (EBM) insist that reviewers search thoroughly for all evidence relevant to a clinical question. Yet when reviewing the subject of EBM itself, these same proponents seem curiously blind to criticism.” I hope to subject EBM to the same skeptical review that this wonderful site gives to a variety of other topics. The letter provides a lot of additional references.

I do not and will not argue that EBM is worthless, just that it has limitations and biases that we must be aware of if we are really skeptics. In my own opinion, some of the issues with EBM are exacerbated when powerful parties (private insurers) can use it as an additional reason to limit the application of clinical judgment by physicians.

Comments are closed for this entry.