Hierarchy of Evidence
Not all evidence is created equal. Below is the basic pyramid of the hierarchy of evidence. At the top of the pyramid is Systematic reviews and meta-analyses of randomized controlled trials, followed by randomized control trials, cohort studies, case-control, case series, and expert opinion and editorials at the bottom.
Systematic Review is a thorough and comprehensive search of the medical literature to answer a specific question. A good systematic review just like any other research should have a method of literature search and data abstraction set a priori with specific search strategy, process of selection, and inclusion/exclusion criteria for reviewed studies. Like all of evidence-based medicine, it should seek to answer a specific answerable question. A systematic review may contain a meta-analysis (see below) to help answer the question, however based on the available evidence a meta-analysis may not be able to completed.
In contradistinction to Systematic Review, non-systematic reviews or literature reviews are much less rigorous and not part of the pyramid of evidence. They are not necessarily based on a single question, but may be an overview or a series of questions on a topic. There is not necessarily a protocol or a specific objective and the results may not be summarized based on specific criteria or quality of evidence.
A Meta-Analysis is a specific type of systematic review in which a statistical tool is employed to pool available results from several similar studies into a single combined result. Thus, all meta-analyses are systematic reviews, but not all systematic reviews contain a meta-analysis
- Thought of as the highest level of evidence.
- Pools multiple studies together to increase the power of the evidence
- Consolidates best evidence available on a focused topic
- Only as good as the available RCTs on the topic (garbage in - garbage out)
- Different studies on the same topic are difficult to reliably statistically pool into one dataset
- "Well Done" studies may be left out because they were did not meet the inclusion/exclusion criteria of the review
- Must be continuously updated as new literature in the area emerges
Randomized Controlled Trial
A randomized controlled trial is a trial that seeks again to answer a specific question by controlling as best as can be controlled for known variables and comparing results of an experimental group with a control group. Patients are enrolled in a randomized fashion to the experimental arm or the control arm. In general, the patients in the experimental arm receive the treatment being studied and standard care while the control arm receives placebo and standard care. This is the highly sought after evidence just below systematic reviews. If well done, it should give the clearest picture regarding the variable of interest.
- If well done, focuses on a single variable and should give a good picture of how that variable affects care
- Clarity of study design
- Well done studies often need financial support, usually from interested parties, which may cloud the reliability of the results
- Difficult to complete a well done study attending to all the potential variables at play
A Cohort Study evaluates the effect of a specific exposure (ie smoking) to find outcomes linked to that exposure. This design takes similar groups of patients (one exposed and one not exposed) and follows them forward in time to evaluate the effects of specific exposures on outcomes. The most famous cohort study is the Framinham Cohort study, which has elucidated much of what we know about cardiovascular risk over time and the classic cardiovascular risk factors.
While this may not be as powerful of a design as an RCT, an RCT cannot always be completed for every type of question. For example, we can’t randomize people to a smoking group and a non-smoking group. Cohort studies are relatively cheap compared to an RCT and thus can be carried out over many years. Of note, a cohort study can be done in a retrospective fashion accessing archived records, however the cohort study still starts with an exposure and follows the groups forward through time.
As we develop large databases of clinical information in informatics, we may be able to leverage the retrospective cohort design to a considerable degree.
- Relatively inexpensive compared to RCTs
- Can be carried out over a long period of time to evaluate the effect of an exposure
- It can be difficult to match cohorts and isolate variables
- It may take a long time to define a cause/effect relationship
A Case Control study, on the other hand, starts with a disease (i.e. cancer) and looks back in time for exposures that may have led to the disease or outcome. Its weakness is in the potential for recall bias in that we are asking patients to look back at all the many things they could have been exposed to. This design is particularly powerful though when an outcome takes a long time to develop or is very rare. There are some very powerful examples of well-done case control studies such as the identification of the link between smoking and lung cancer.
- Useful for finding the cause of rare diseases
- Relatively inexpensive compared to Cohort and RCTs (this cost may significantly increase if direct patient/family interviews are required to find exposures)
- Significant risk of being confounded by recall bias
- Often difficult to elucidate individual exposures and define cause/effect
A Case Series is a report of several observed cases or a collection of patients treated in a similar manner. There is no rigorous study design and no significant limitations on how this type of study is reported. Its goal is not necessarily to define a specific cause and effect relationship but to bring about awareness of possible associations. A case series may be written in order to highlight a possible link to spur further research on the topic. Other times a case series is published is because there is no other literature out there on a specific topic and this topic may be very rare. This is often the case for things like the effects of toxicological exposures or treatments for toxicologic exposures.
A case report is a report of a single observation. This may be as simple as an interesting or notable case or it may be a finding where an association is suspected but there have been no other reports of this association in the literature.
Other Trial Designs
Of note, there are other trial designs not covered here, but these are in general the trial designs best evaluated using the tools of evidence based medicine. Higher on the pyramid could be listed: systematic reviews of systematic reviews and N-of-1 trials. Also below case control studies could be listed cross sectional studies which look more generally at a population to define prevalence.
Submitted by Jason T Schaffer, MD