Healthcare worker sitting at laptop researching
HealthJanuary 26, 2021

Appraising experimental research to determine the level of evidence

By: Sarah Handzel, BSN, RN
As all nurses know, experimental research is a huge part of nursing practice. Research shapes policies and procedures that determine patient outcomes.

But it can be difficult to determine what qualifies as “strong” evidence when evaluating research. A recent article in Nursing 20211, the first of a three-part series, helps nurse leaders assess the quality, quantity, and consistency of research evidence to help determine its strength.

Three elements of strong research

According to the Agency for Healthcare Research and Quality, the strength of any research evidence must be determined using the following three elements:

  • Quality – Perhaps the most challenging element nurses must evaluate, quality refers to the research methods used to ensure valid research results which are not influenced by bias or occurring by chance.
  • Quantity – There must be a sufficient number of studies on the impact of a particular topic or treatment, and the studies must be of appropriate size.
  • Consistency – Similar findings to the research study should be reported from other sources.

Evidence hierarchies: Understanding different approaches to research

Research evidence hierarchies can be thought of as pyramids made up of five separate levels. At the top of the pyramid, which can be thought of as Level 1, the evidence is strongest. The bottom of the pyramid, or Level 5, contains the weakest level of evidence for any given research topic.

Level 1: RCTs, systematic reviews, and meta-analyses

The strongest evidence for or against a new treatment or procedure is represented in Level 1, which evaluates the results from randomized controlled trials (RCTs), systematic reviews, and meta-analyses. In RCTs, studies must meet three criteria:

  1. Random assignment of study participants into two or more groups
  2. A treatment or procedure given to at least one of the groups
  3. A control group that doesn’t receive the new treatment or procedure

Systematic reviews use meticulous processes to identify, appraise, and synthesize evidence on a particular topic. Meta-analyses take this a step further by conducting statistical analyses of the synthesized data. This allows researchers to obtain statistics representing the effect of the treatment or procedure across multiple studies.

Level 2: Quasi-experimental research

The research results contained in Level 2 are termed “quasi-experimental” because, in general, the studies supplying the results lack one or two of the three criteria required for a true RCT design. Systematic reviews may be included in this level with or without meta-analysis of the data. There is still manipulation of study participants or the introduction of a new intervention, such as a new treatment.

Nurses should be more cautious when evaluating Level 2 experimental results, since reviews that include quasi-experimental research aren’t as strong as those including RCTs. But quasi-experimental research will still continue—in most cases, these types of studies are simpler to carry out.

Levels 3-5: Nonexperimental factors

The strength of research evidence continues to decline through levels three through five. Level 3 may be made up of nonexperimental study results which may or may not feature systematic reviews and meta-analysis. Level 4 is generally based on expert opinions which are, in turn, based on scientific evidence, practice guidelines, or consensus panels. Finally, Level 5 evidence usually comes from literature reviews or case reports, quality improvement programs or financial evaluations, or expert opinions based on experimental evidence.

The next article in this three-part series delves into the lower levels of the evidence hierarchy pyramid in greater detail. Still, nurses must rely on a sufficient number of sources to ultimately arrive at conclusions which influence nursing practice. It isn’t enough to find one piece of evidence that supports their suspicions—instead, critical thinking skills must be applied to determine the strength of evidence based on quality, quantity, and consistency of study results.

Sarah Handzel, BSN, RN
Freelance Health and Medical Content Writer, Wolters Kluwer Health
Sarah has over nine years’ experience in various clinical areas, including surgery, endocrinology, family practice, and pharmaceuticals. She began writing professionally in 2016 as a way to use her medical knowledge beyond the bedside to help educate and inform healthcare consumers and providers.
  1. Glasofer, Amy DNP, RN, NE-BC, and Ann B. DRNP, RN, ANP-C, CNS-C Townsend. “Determining the Level of Evidence: Experimental Research Appraisal.” NursingCenter, 2021, nursingcenter.com/journalarticle?Article_ID=5738006&Journal_ID=54016&Issue_ID=5737721.
Solutions
Lippincott Solutions
Our best-in-class suite of evidence-based, institutional software can help you to balance clinical and business needs by streamlining workflow, standardizing care, and improving reimbursable patient outcomes.