Prompt 6
WSM2: Evidence-Based Innovation Proposal Evidence-Leveling Navigation Tool
Evidence-leveling Navigation Tool
Introduction
The importance of leveling
As an advanced practice nurse, why is it important to understand the strength of the research on which to base organizational decisions regarding policy and procedure change? To answer this question, we must understand that not all research is good research, and there must be a clear way to differentiate this. Ranking evidence (or leveling) is based on the study's design, methodology, quality, and applicability to answer the question or issue presented in practice. Decision-making must be primarily predicated on the highest levels of evidence that are available to practice. While lower levels of evidence shouldn’t be disregarded, decision-making should primarily be predicated on the strongest evidence available to support practice change.
Why is a leveling tool important?
Many published tools are available for use by researchers and practitioners. These tools offer a standardized method to review articles to determine the quality of the research or evidence presented in the article. An organization must adopt a standardized, published tool to enable staff to critically examine the literature on which policy and procedure changes will be supported by solid evidence, hence, evidence-based practice.
How do you start the leveling process:
· Review the title of the article for possible clues to the hierarchy (i.e., meta-analysis)
· Review the abstract for a summary of the methods or design of the study
· Identify the research question or purpose of the study/article
· Review the methods section (if there is one)
· Do they say it is a study or a review?
· How do they describe the participants?
· What methods did they use to collect data (i.e., surveys, intervention, interviews)?
· Was there IRB approval for the study (no IRB for a review)?
· Carefully review this guide's Research vs Expert Opinion section.
· From there, you will review each of the sections of the guide to determine the appropriate level and hierarchy for your articles.
Table of Contents
Evidence-leveling Navigation Tool 1
Why is a leveling tool important? 1
How do you start the leveling process: 1
Example of a Level VII, Expert Opinion: 4
Examples of high-level reviews: 7
Example of a low-level review: Level VII, Expert Opinion 7
Single, Quantitative Studies 8
Level II, Randomized Control Trials (True Experimental) 8
Level III - Quasi-Experimental Study 9
Quantitative Studies, continued 10
Level VI, Qualitative Study 12
Example of a qualitative research study: 12
For further understanding of Qualitative Research Designs, refer to Chapter 22: 12
Research vs Expert Opinion
A critical first step in determining the strength and hierarchy of an article is to determine if it is a research study or an expert opinion (non-research). It will be important for you to review your article and search for the clues in the article that will help you decide if it is true research or expert opinion (non-research).
Research:
· A true research article can be a single research study
or
a high-level review of many research studies on the same topic.
· A single study can be either a Level IV (Non-experimental), a Level III quasi-experimental, a Level II randomized control trial, or a Level VI, single qualitative study.
· A mixed-method study may involve both quantitative and qualitative research in a single study. Always level the article at the highest level. For example, a study has used a survey to collect quantitative data and open-ended questions to collect qualitative data. This study would be leveled as a Level IV, non-experimental study instead of a Level VI, qualitative study.
· A high-level review can be a Level I integrative review, Level I systematic review, a Level I meta-analysis, Level I National Practice Guideline or a Level V meta-synthesis.
Characteristics of a single study : The article will contain
· a research question, study aim, purpose statement, and/or hypothesis.
· a methods section that discusses the study's design, how data is collected, how the sample is determined, and how the data is analyzed. The author(s) often clearly articulate that they conducted a single study.
· a description of how the sample participants were selected. The article may use terminology describing a convenient or random sample.
· a description of the IRB (Institutional Review Board approval) process if human subjects/participants were involved in the research. IRB approval will not be needed or indicated if the study does not directly involve human participants. Articles may refer to IRB approval as ethical approval or review by an ethics committee. If IRB is present, it generally is within the article's methods section or may be found at the end of the article, just above the reference list.
Characteristics of a high-level literature review: The article will contain
· a research question, aim, or purpose statement.
· a methods section that discusses the detailed description of how the literature review was conducted. It will often include the databases used, the search terms used, inclusions/exclusion criteria for the articles chosen, the number of articles found, and the final number of articles selected to review. The methods section may describe the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) process, a highly rigorous process in high-level reviews.
· a table that describes each article used in the review. This information is important as it will assist you in determining if the individual studies are quantitative and/or qualitative.
· immense detail, followed by substantial conclusions synthesized and drawn from the reviewed articles based on factual findings.
Expert Opinion
Level VII, Expert Opinion
Articles that reflect the author’s expert view on a specified topic or concept are considered expert opinions.
Examples of expert opinions are:
· Editorials.
· Case studies.
· Position statements from an organization.
· Papers that explore a particular topic or concept.
· Quality improvement projects.
· Institutional practice guidelines. A specific institution’s practice guidelines.
· Narrative literature reviews.
· Scoping Reviews.
· Rapid reviews.
Characteristics of an expert opinion: Expert Opinions
· are not research.
· do not follow the research process and often do not include a methods section, sample, analysis, or results section.
· focus on process improvement at a local organization/institutional level that is not generalizable to the broader healthcare arena.
· include QI projects which can appear to be a single study that follows the research process and involves human subjects but does not discuss IRB approval or state that IRB/ethics approval was waived. However, because QI projects are designed to improve a known problem, rather than answer a research question, they are exempt from IRB.
· can be low level literature reviews that lack the rigor of a high-level review and do not include a description of a robust search method and article selection criteria.
Example of a Level VII, Expert Opinion:
Luna, J. A. & Behan, D. F. (2024). Increasing nurse confidence through high-fidelity
simulation. MEDSURG Nursing, 33(1), 24-50. doi:10.62116/msj.2024.33.1.24 https://research.ebsco.com/c/25xrgu/search/details/i2f6utklun?limiters=&q=increasing%20nurse%20confidence%20through%20high%20fidelity%20simulation
Reviews
High-Level Reviews:
· Level I Integrative review
· Level I Systematic review
· Level I National Practice Guidelines
· Level I Meta-Analysis
· Level V Meta-Synthesis
Low-Level Review:
· Level VII, Expert Opinion
Characteristics of a high-level review: High-level reviews
· will never require or indicate Institutional Review Board (IRB) or ethics committee approval as they do not directly include human subjects.
· will begin by formulating the research question (purpose or aim) that will be answered by the review.
· are time-consuming to complete and will often involve a team of researchers.
· may identify the use of a rigorous, standardized review protocol such as PRISMA.
· will include a methods section where a detailed protocol is identified to guide the literature search.
· Several search engines or databases involved in the search are identified.
· The time frame for the search is identified.
· Search terms used in the search are defined.
· Inclusion and exclusion criteria are identified.
· The final number of articles will be narrowed as the inclusion and exclusion criteria are applied.
· There is no requirement for the number of articles, but most systematic reviews include around 10-60 articles.
· The final studies selected for the review are usually placed in an appraisal table that contains pertinent information from each article, such as the type of study, methodology, and results.
· The researchers extract the data from the final studies and create an analysis of the results that conveys a deeper understanding of the body of evidence.
· may be considered an integrative review if the articles reviewed include both quantitative and qualitative research. They may also include non-research articles and reviews.
· may be considered a systematic review if all articles reviewed are quantitative research.
· may be considered a national practice guideline if a systematic review has been endorsed as a practice guideline by a national organization such as the National Association of Orthopaedic Nurses (NAON), American Association of Critical-Care Nurses (AACN), or Association of Operating Room Nurses (AORN).
· may be considered meta-analysis if all articles reviewed are quantitative research and include several randomized control trials (RCTs) that are statistically re-analyzed for further conclusions. Since this is a very prestigious type of review, it will often be identified in the article's title.
· may be considered a meta-synthesis if all articles reviewed are qualitative research.
Examples of high-level reviews:
Integrative review
Taylan, S., Özkan, I., & Yavuz van Giersbergen, M. (2024). Nursing students'
operating room experiences: A qualitative metasynthesis. Journal of
PeriAnesthesia Nursing, 39(5), 892-901. https://doi.org/10.1016/j.jopan.2023.12.029
Meta-Analysis
Grygorian, A., Montano, D., & Shojaa, M. (2024). Digital health interventions and
patient safety in abdominal surgery: A systematic review and meta-analysis. JAMA Network Open, 7(4), e248555. https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2818102#google_vignette
Characteristics of a low-level review: Low-level Reviews
· lack a robust methods or methodology section with defined search parameters.
· will identify multiple articles that will be used to define or explore a concept or topic.
· will discuss an identified topic or concept
· may be identified as a literature review, narrative review, scoping review, or rapid review
· will be considered a Level VII, Expert Opinion
Example of a low-level review: Level VII, Expert Opinion
Alhur, A. A. (2024). Impact of technological innovations on healthcare delivery: A
literature review of efficiency, patient care, and operational challenges. World Journal of Biology Pharmacy and Health Sciences, 18(2), 216-219. https://doi.org/10.30574/wjbphs.2024.18.2.0273
Single, Quantitative Studies
Experimental Studies
When determining whether a study is experimental (either a true experimental or quasi-experimental), it is helpful to substitute the words “intervention” or “implementation” for the word “experimental”. In other words, did the researcher introduce an intervention or implement a change to determine if there was an effect of the intervention or change on the sample studied? If an intervention or change is implemented, then the study is an experimental study as long as there is IRB or ethics board approval.
Level II, Randomized Control Trials (True Experimental)
A Randomized Control Trial (RCT) is single, quantitative study with a true experimental design. Often, the authors of RCTs include this language in the title of the article and in the abstract. If this is the case, review the methodology to be certain it is an RCT based on the criteria below.
Characteristics of a Randomized Control Trial (RCT) : Randomized control trials
· are rigorous, time-consuming, highly respected research studies.
· must include IRB approval if human subjects are involved.
· always include an intervention or implementation that is described in the methods section of the article.
· will always include these three hallmarks:
· A Random Sample: This means that each participant or element has an equal chance of being placed into the control group and/or intervention group. If a random sample is not identified and described within the methods section of the article, it is not a level II RCT.
· A Control Group: This is the group that did not receive the intervention or implementation prior to testing or measurement. g
· An Intervention Group: This is the group that received the intervention or implementation. The group is “tested,” and the results are compared to the scores of the control group to see if the intervention/implementation had a significant positive or negative effect.
Example of a True Experiment or Randomized Control Trial:
Budak, M.A. and Geckil, E. (2024). The effect of home environment modification
nursing intervention on symptom control, quality of life, and number of triggers in children with allergic rhinitis: A randomized control trial. Journal of Pediatric Nursing, 74, e28-e37. https://doi.org/10.1016/j.pedn.2023.10.038
For further explanation of Experimental research designs:
Flanagan, J. & Beck, C. T. (2025). Quantitative research design . Polit & Beck’s Nursing
Research (12th ed), pp. 175-204. Wolters Kluwer.
https://wgu.vitalsource.com/reader/books/9781975223830/epubcfi/6/44[%3Bvnd.vst.idref%3DFlanagan12e9781975223830-ch009]!/4
Level III - Quasi-Experimental Study
A Quasi-Experimental single quantitative study is considered an experimental study, but it is missing at least one of the three hallmarks of a true experimental design. Either random sampling, the control group, or the intervention group is missing. Randomization or a random sample is generally the missing element.
Characteristics of a quasi-experimental study . Quasi-Experimental studies
· must include IRB approval if human subjects are involved
· may identify a single group that is tested before and after the intervention
· may identify two or more non-randomized groups, with one serving as a control group
· will describe a convenience sample rather than a random sample
· may include retrospective (historical) data i.e. pre-existing chart data or government data sets.
· may use the terms pre-test/post-test, pre-intervention/ post-intervention, before/ after, or baseline/ post-implementation
Example of a Quasi-Experimental Study
Torné-Ruiz, A., Reguant, M., & Roca, J. (2023). Mindfulness for stress and anxiety
management in nursing students in a clinical simulation: A quasi-experimental study. Nurse Education in Practice, 66. https://doi.org/10.1016/j.nepr.2022.103533
For further explanation of Quasi-Experimental research designs:
Flanagan, J. & Beck, C. T. (2025). Quantitative research design . Polit & Beck’s Nursing
Research (12th ed), pp. 175-204. Wolters Kluwer.
https://wgu.vitalsource.com/reader/books/9781975223830/epubcfi/6/44[%3Bvnd.vst.idref%3DFlanagan12e9781975223830-ch009]!/4
Quantitative Studies, continued
Level IV, Non-Experimental
Non-experimental, single studies collect quantitative (numerical) data.
Characteristics of a non-experimental study . Non-Experimental studies
· are true research but do not have an experimental design.
· must include IRB approval if human subjects are involved.
· do not include an intervention or implementation of any kind. This infers that the researcher is a bystander and does not intervene or have any part in the study other than to collect the data, analyze the data, and report the findings.
· may be referred to as observational research.
· may use quantitative data collection methods such as:
· Surveys using Likert scales
· Published tools, instruments, or scales
· Pre-existing data (retrospective data sets)
Examples of non-experimental studies include:
· retrospective studies using past data.
· case studies.
· cross-sectional studies where the data is collected at one point in time from many individuals.
· cohort comparison studies where the participants are similar.
· descriptive studies that describe a population, situation, or phenomenon.
· prospective studies to follow a group of participants over time to look at outcomes.
· correlational single studies that describe a relationship between two variables.
· comparison studies that observe the effect of different treatments (variables) in healthcare on different groups
Examples of Level IV, Non-Experimental Studies:
Lundemose, S. B., Laugesen, C., Ranjan, A. G., & Nørgaard, K. (2023). Factory-
calibrated continuous glucose monitoring systems in Type 1 Diabetes: Accuracy during in-clinic exercise and home use. Sensors, 23(22), 9256. https://doi.org/10.3390/s23229256
For further explanation of experimental and non-experimental research designs,
Flanagan, J. & Beck, C. T. (2025). Quantitative research design . Polit & Beck’s Nursing
Research (12th ed), pp. 175-204. Wolters Kluwer.
https://wgu.vitalsource.com/reader/books/9781975223830/epubcfi/6/44[%3Bvnd.vst.idref%3DFlanagan12e9781975223830-ch009]!/4
As a reminder, a mixed-method study may involve quantitative and qualitative research. Always level the article at the highest level. For example, a study has used a survey to collect quantitative data and open-ended questions to collect qualitative data. This study would be leveled as a Level IV, non-experimental study instead of a Level VI, qualitative study.
Qualitative Research
Level VI, Qualitative Study
Qualitative single studies collect non-numerical data. Qualitative researchers believe that all experience is relative and unique to the individual; in other words, human experience cannot be measured quantitatively or statistically.
Characteristics of qualitative research: Qualitative studies
· must include IRB approval because human subjects are always involved.
· will include subjective data that focuses on the perspective of the informant or research participant.
· will use qualitative research methods such as
· semi-structured interviews.
· focus groups.
· content analysis of narratives.
· open-ended interviews.
· participant observation.
· may include data that describes the characteristics or demographics of the sample (participants) i.e. gender, race, level of education.
· will include all aspects of the research process, including the research question/aim/purpose, methods design, sampling plan, consent and protection of participants (IRB), data collection, and data analysis.
· report qualitative data from the participant’s words (quotes).
· will apply thematic analysis to identify predominant themes in the data. Researchers will provide participant quotes to validate these themes.
As a reminder, a mixed-method study may involve quantitative and qualitative research. Always level the article at the highest level. For example, a study has used a survey to collect quantitative data and open-ended questions to collect qualitative data. This study would be leveled as a Level IV, non-experimental study instead of a Level VI, qualitative study.
Example of a qualitative research study:
Davies, N., Aker, N., West, E., Rait, G., & Sampson, E.L. (2024). Acceptability of using
a decision aid to support family carers of people with dementia towards the end of life: A qualitative study. Health Expectations, 27(3), e14123. https://doi.org/10.1111/hex.14123
For further understanding of Qualitative Research Designs, refer to Chapter 22:
Flanagan, J. & Beck, C. T. (2025). Qualitative research design and approaches. Polit &
Beck’s Nursing Research (12th ed), pp. 462-486. Wolters Kluwer. https://wgu.vitalsource.com/reader/books/9781975223830/epubcfi/6/72[%3Bvnd.vst.idref%3DFlanagan12e9781975223830-ch022]!/4
Western Governors University 1 | Page
Accessibility Review Completed and Approved for Distribution and Use.
Academic Product Management [HC - Based on review completed 09/13/24.]
Western Governors University Accessibility Review Completed and Approved for Distribution and Use. Academic Product Management [HC - Based on review completed 09/13/24.]