Hard Data Doesn't Represent the Best Medicine

An article in the late February issue of Time Magazine on evidence-based medicine and why it might be a bad thing for doctors to fully rely on it reminded me of what has to be my hands-down favourite journal article, ever. Now four years old, it's critique of evidence-based medicine is still one of the sharpest I've ever seen. From the BMJ website: Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials.

A Dr. Gordon Smith and Ms. Jill Pell decided to "determine whether parachutes are effective in preventing major trauma related to gravitational challenge," utilizing prominent sources to analyze the data available, sans any actual randomized trial. Their conclusion is an argument I've heard against evidence-based medicine many times now, but never quite so succinctly as this:

As with many interventions intended to prevent ill health, the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials. Advocates of evidence based medicine have criticised the adoption of interventions evaluated by using only observational data. We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute.

- Kelly Hills
[ed: please welcome guest blogger Kelly Hills, student in the Alden March Bioethics Institute MS in Bioethics program and doctoral student in the AMBI joint degree program being "taught out" by Albany Med/UGC and UAlbany Department of Philosophy; Kelly blogs for the Women's Bioethics Project blog as well as keeps her own blog about 'academia as an extreme sport', chronicles of the [mis]adventures of an academic in training to work in bioethics. She is also working at AMBI in the Scholar slot and recently co-authored a Nature Medicine review essay on transplantation in the black communities of America. Welcome Kelly, who joins Stuart Rennie, John Robertson and others along with your editors.]

posted Mar 31, 2007 | link | jump to the comments (4)

comments

Interesting. I'm quite interested in EBM, and I'm not at all sure I find the BMJ critique compelling. The most basic question is what, after all, is the alternative? Should we advise clinicians not to seek the best available rigorous evidence as a guide to assessment and therapeutic intervention?
That seems unwise to me. If we think that practice ought to be guided by such evidence, not to the exclusion of ethical and clinical intuition, but in addition to it, then the remaining questions are epistemological in nature. I tend to think such questions are frequently among the most intractable; why do you think I am so sympathetic to skepticism?
My own view is that expressing a commitment to practicing EBM is simply expressing a commitment to engage these difficult interpretive and evidentiary questions. What counts as good evidence? How do we rank and assess evidence? These questions are not new, IMO. Hume's perspective on it is fascinating, and Popper was so troubled by the ineluctable uncertainty of induction that he questioned whether truth conditions could even be applied to inductive propositions.
And I've mentioned on our little blog that the reflexive reliance on RCTs to resolve all questions is a major problem with EBM, one that is fortunately neither unknown and nor unchallenged even by proponents of EBM.
I rather like Brody, Miller, and Bogdan-Lovis' perspective that EBM has more to fear from its friends than its enemies. Evidentiary problems are immensely difficult, IMO. But abandoning any commitment to engage the hermeneutic enterprise risks far greater harms, IMO. As important a marker of significance as intuition may be, there are far too many narratives in the history of Western medicine in which our intuitions were badly mistaken on conceptions of disease and therapeutics. Like Russell once remarked, common sense is stone age metaphysics.
All of the above, JMO.
Welcome to the blog, K!

- by Daniel on Apr 1, 2007 at 7:17 PM | link

Daniel,
I think that Smith and Pell neatly articulate the main critique of EBM, which I do share (and the Time article also notes): sometimes, it's a bit too dogmatic for its own good. I tend to shy away from the dogmatic only because I get really irritated when asked to think exclusively in black and white - life is more than that, or shades of grey, we have an entire Crayola box at our disposal.
I think my concern largely comes from two things. The first, the Time article neatly sums up:

"I'm worried about training a generation of physicians who don't have the other skills they need for the optimal practice of medicine," says Dr. Mark Tonelli, a pulmonary-care specialist at the University of Washington in Seattle. "They can read the scientific literature, understand the statistics, but they don't understand how that should influence their treatment of the individual in front of them." What's more, some insurance companies have been very aggressive in using evidence-based arguments to deny payment for untested treatments--a circular problem, because how do you create the evidence the insurers demand unless you test the untested?

The second concern is a bit more abstract, and to be honest I'm feeling entirely too lazy to move out of the chair and up the stairs to find the paperwork I've been collecting on this. But essentially, it's linked to the above concern by Tonelli, and is about the attitudes people enter the medical profession with, and how that affects their education. Strong beliefs, once they begin to seem like accepted truths, affect the assumptions and questions that people, scientists, doctors, ask.
EBM, at least those who practice the rather firm notion of it, seems to me that it's attempting to put every patient through an equation - if A plus B equal C, then treatment is F. And while sometimes this might work, other times notsomuch - I think the human body is simply too complicated to be so neatly reduced in the manner EBM threatens.

- by Kelly on Apr 3, 2007 at 4:44 AM | link

Hey Kelly,
That all is fair, but that just seems to evoke Brody et al's point. The problems you describe are not reasons that inhere in EBM, but in the ways in which the interpretive commitments are engaged.
The dogmatic application of "evidence" in the search for total objectivity is indeed problematic for many reasons, not least of which is that it is inconsistent with the very raison d'etre for EBM in the first place, namely, the inexorable uncertainty of any inductive enterprise. It is no real response to the problem of uncertainty to assert that we can eliminate it entirely.
But all of that is just to say that not all who profess to practice EBM engage interpretive commitments and perspectives that proponents of EBM ought to endorse. It doesn't follow from this legitimate critique that providers should abandon any commitment to dispense care in ways that are consistent with the best available evidence.
And again, I certainly don't think stripping clinical practice of its human, variable dimensions, or of the role of clinical and ethical intuition, is a necessary byproduct of avowing a commitment to EBM. That this has seemed to happen says more about our willingness to tolerate ambiguity and uncertainity and much less about the validity of trying to assess evidence in the hopes of guiding practice.
I continue to have difficulty in seeing any viable alternative.

- by Daniel on Apr 3, 2007 at 2:26 PM | link

*grins*
The alternative, Daniel, is what there always is when you're discussing extreme positions: moderation.
EBM is obviously valuable, but we shouldn't rely on it exclusively - just as we shouldn't rely exclusively on intuition.
Balance and moderation, instead of dogmatic adherence.

- by Kelly on Apr 4, 2007 at 2:45 AM | link