Evidence Based Medicine is Not a Stool

Evidence Based Medicine

Meira (2020)


Evidence based medicine (EBM) exists to de-emphasize intuition, unsystematic clinical experience, and pathophysiologic rationale as sufficient grounds for decision making. However, bias is still unavoidable, even in the literature. Two types of methodological fallacies are HARKing and p-hacking.

HARKing: hypothesising after results are known. This is when researchers take a look at the data and fit an idea into it. This is fine in an exploratory way, but results in a lot of false-positives. Researchers should hypothesise before the data comes in. 

P-hacking: this refers to altering statistical methods, looking at too many dependent variables, or even leaving out data to make sure the p-value drops below 0.05. A great quote from Ronald Coase is “If you torture the data long enough, it will confess to anything.” This too will obviously result in a higher likelihood of false-positives.

Fig1 meira ebm
From: Meira (2020), OSF

Wrong Analogy

The most common analogy of EBM is a three-legged stool (see figure). This concept is wrong since it weights the three pillars equally. In reality, these concepts are weighted differently. First, we take all available evidence, then we use our critical appraisal skills to evaluate the evidence, and at last, we explain the available options as clearly as possible to the patient — who will make the final decision. EBM is a funnel, rather than a stool (see figure). 

It is impossible for clinicians to be completely up-to-date on the literature of every topic they encounter in clinical practice. This is where systematic reviews, practice guidelines, and consensus statements come in.

Fig2 meira ebm
From: Meira (2020), OSF

What About my Experience?

What clinicians see in the clinic is also evidence — to the surprise of many. However, this information is gathered in an uncontrolled and biased way. Meaning the weight we give it, will be rather low. Certainly in comparison with high-quality trials and the abovementioned forms of synthesized evidence. The key is combing all of these information sources. 

“The literature cannot refute what is seen in the clinic, but it can refute our explanations of those experiences.”

Immediate positive effects from modality X are real. However, the evidence can help us identify the reason for progression since we as humans are unable to do this unbiased, controlled, and statistically. It might very well be that the progression is merely a regression to the mean, natural history, placebo, the non-specific effects of attention and/or the face-validity of the intervention in the eyes of the patient. This does not invalidate the experience but puts it into context and provides a deeper understanding of potential mechanisms and context. 

What You Should Do

The clinician should provide dispassionate, complete, and balanced information to the patient. At the end of the road, the patient chooses what is best from a personal standpoint. They know what the mechanisms are (even if it is likely placebo), the costs, time-frames, risks, discomfort, etc. This way, a final decision can be made.

EBM is a tool to process information into a digestible format for patients. It is a multistep process for clinicians to go through. Think funnel, not stool.


Meira (2020)

Phy arrow right