Systematic Reviews & Guidelines

CHAPTER 13

Building an Evidence-Based Nursing Practice

Research evidence has greatly expanded over the last 30 years as numerous quality studies in nursing and other healthcare disciplines have been conducted and disseminated. These studies are commonly communicated via conferences, journals, and the Internet. The expectations of society and the goals of healthcare systems are the delivery of quality, safe, cost-effective health care to patients, families, and communities (Sherwood & Barnsteiner, 2017; Straus, Glasziou, Richardson, Rosenberg, & Haynes, 2011). To ensure the delivery of quality health care, the care must be based on the current best research evidence available. Over the last 15 years, nursing programs have provided students with knowledge about evidence-based practice (EBP) to encourage graduates to base their practice on current research. The emphasis on EBP in nursing education programs and clinical agencies has improved outcomes for patients and families, nurses, and healthcare agencies (Mackey & Bassendowski, 2017; Melnyk, Gallagher-Ford, & Fineout-Overholt, 2017).

Evidence-based practice (EBP) is an important theme in this text that was defined in Chapter 1 as the integration of the best research evidence with nurses’ clinical expertise and patients’ circumstances and values in the delivery of quality, safe, and cost-effective health care (Straus et al., 2011). Best research evidence is produced by the conduct and synthesis of numerous high-quality studies in a selected health-related area. This chapter builds on previous EBP discussions in this text to provide you with strategies for implementing the best research evidence in your practice and moving the nursing profession toward EBP.

The benefits and challenges associated with EBP are described to increase your understanding of evidence-based nursing practice. A format is provided for developing clinical questions to direct your searches for existing research-based evidence to use in practice. Guidelines are provided for critically appraising research syntheses (systematic reviews, meta-analyses, meta- syntheses, and mixed-methods systematic reviews) to determine the knowledge that is ready for use in practice. Two nursing models that have been developed to facilitate EBP in healthcare agencies are introduced. Expert researchers, clinicians, and consumers—through government agencies, professional organizations, and healthcare agencies—have developed an extensive number of evidence-based guidelines. A framework for reviewing the quality of these evidence-based guidelines and for using them in practice is provided. This chapter concludes with a discussion of the nationally designated EBP centers and translational research implemented to promote evidence-based health care.

Benefits and challenges related to evidence-based nursing practice

EBP is a goal for the nursing profession and each practicing nurse. At the present time, some nursing interventions are evidence-based, but many interventions require additional research to generate essential knowledge for making changes in practice. Some clinical agencies are supportive of the EBP process, and others are not. This section identifies some of the benefits and challenges associated with implementing evidence-based nursing.

Benefits of Evidence-Based Nursing Practice

The greatest benefits of EBP are improved outcomes for patients, providers, and healthcare agencies (Melnyk et al., 2016; Moorhead, Johnson, Maas, & Swanson, 2013). Agencies and organizations nationally and internationally have promoted the synthesis of the best research evidence in thousands of healthcare areas by teams of expert researchers and clinicians. These research syntheses, such as systematic reviews and meta-analyses, have provided the basis for developing strong evidence-based guidelines for practice. These guidelines identify the best treatment plan, or the gold standard for patient care, in selected areas to improve patient outcomes. Students and clinical nurses have electronic access to numerous evidence-based guidelines to assist them in making the best clinical decisions for their patients. These evidence-based syntheses and guidelines are available nationally and internationally and can be easily accessed online through different institutions, such as the National Guideline Clearinghouse (NGC, 2017a) in the United States, the Cochrane Collaboration (2017) in the United Kingdom, and the Joanna Briggs Institute (2017) in Australia.

Some chief nurse executives (CNEs) and healthcare agencies are highly supportive of EBP, as indicated by their attitudes and provision of resources to support EBP (Melnyk, Fineout-Overholt, Giggleman, & Choy 2017). Leaders in these clinical agencies recognize that EBP promotes quality outcomes, improves nurses’ satisfaction, and facilitates achievement of accreditation requirements. In a national study of CNEs, Melnyk et al. (2016) found that an organization with an EBP culture of conducting and using research evidence in practice had substantial improvements in several patient outcomes. The Joint Commission (2017) revised their accreditation criteria to emphasize patient care outcomes achieved through EBP.

Many CNEs and chief nursing officers (CNOs) are trying to obtain or maintain Magnet status, which documents the excellence of nursing care in healthcare agencies. Approval for Magnet status is obtained through the American Nurses Credentialing Center (ANCC), and the national and international healthcare agencies that currently have Magnet status can be viewed online (ANCC, 2017). The Magnet Recognition Program® emphasizes EBP as a way to improve the quality of patient care and revitalize the nursing environment. Clinical agencies seeking or maintaining Magnet status must document research-related outcomes, including nursing studies conducted and professional publications and presentations by nurses. For each study, the title of the study, principal investigator or investigators, role of nurses in the study, and study status need to be documented in Magnet applications and reports (ANCC, 2017).

The Quality and Safety Education for Nurses (QSEN, 2017) project was implemented to improve prelicensure nurses’ “knowledge, skills, and attitudes (KSAs) that are necessary to continuously improve the quality and safety of the healthcare systems within which they work.” QSEN competencies were developed in six areas essential for students and registered nurses’ (RNs) practice: patient-centered care, teamwork and collaboration, EBP, quality improvement (QI), safety, and informatics. EBP is an important area in your prelicensure education, and educators are assisting students in achieving the following EBP competencies:

• Participate effectively in appropriate data collection and other research activities.

• Adhere to institutional review board (IRB) guidelines.

• Base individualized care plan on patient values, clinical expertise, and evidence.

• Read original research and evidence reports related to area of practice.

• Locate evidence reports related to clinical practice topics and guidelines.

• Participate in structuring the work environment to facilitate the integration of new evidence into standards of practice.

• Question rationale for routine approaches to care that result in less than desired outcomes or adverse events.

• Consult with clinical experts before deciding to deviate from evidence-based protocols (QSEN, 2017).

Educators have changed nursing curricula to include EBP content and added courses that have improved students’ perceptions and confidence in research and EBP (Keib, Cailor, Kiersma, & Chen, 2017). Warren et al. (2016, p. 15) found that “younger RNs with fewer years in practice were more likely to have positive beliefs toward EBP and embedding it into the organization culture.” In working toward EBP, students and practicing RNs are encouraged to embrace the benefits of EBP; critically appraise current research evidence; refine agency protocols, algorithms (clinical decision trees), and policies based on current research; use evidence-based guidelines that are available; and collect data as needed for research projects.

Challenges to Evidence-Based Nursing Practice

Challenges to the EBP movement in nursing have been practical and conceptual. One of the most serious concerns is the limited research evidence available regarding the effectiveness of many nursing interventions. EBP requires synthesizing research evidence from randomized controlled trials (RCTs) and other types of intervention studies, which are still limited in nursing. Systematic reviews and meta-analyses conducted in nursing also are limited when compared with other disciplines, such as medicine and psychology (Cochrane Collaboration, 2017; Gray, Grove, & Sutherland, 2017; NGC, 2017b).

Another challenge is that research evidence is generated based on population data and then is applied in practice to individual patients. Sometimes it is difficult to transfer research knowledge to individual patients, who respond in unique ways or have unique circumstances and values. More work is needed to promote the use of evidence-based guidelines with individual patients. In response to this concern, the National Institutes of Health (NIH, 2017) is supporting translational research (discussed later in this chapter) to improve the use of research evidence with different patient populations in various settings. Patients who have poor outcomes when managed according to an evidence-based guideline need to be reported and, if possible, their circumstances should be published as a case study. Electronic health records (EHRs) make it more feasible to determine patient outcomes of care that have been delivered using EBP guidelines.

Another serious challenge is that some healthcare agencies and administrators do not provide the resources or support necessary for nurses to implement EBP. In their national study, Melnyk and colleagues (2016, p. 9) reported, “Although the CNEs and CNOs stated that their highest priorities were quality and safety, EBP was not listed as a top priority and very little of their budgets were allocated to implementing and sustaining evidence-based care.” Lack of support and resources for EBP included: (1) inadequate access to research journals and other sources of synthesized research findings and evidence-based guidelines; (2) inadequate knowledge or mentoring on how to implement evidence-based changes in practice; (3) heavy workload, with limited time to make research-based changes in practice; (4) limited authority to change patient care based on research findings; (5) limited support from nursing administrators or medical staff to make evidence-based changes in practice; (6) limited funds to support research projects and research-based changes in practice; and (7) minimal rewards for providing evidence-based care to patients and families (Eizenberg, 2010; Melnyk et al., 2016; Melnyk et al., 2017; Straka, Brandt, & Brytus, 2013; Warren et al., 2016). The success of EBP is determined by all involved, including healthcare agencies, administrators, nurses, physicians, and other healthcare professionals. The following content was developed to assist students and RNs in facilitating evidence-based nursing practice.

Developing clinical questions to search for existing research-based evidence for use in practice

Developing a clinical question in an area of interest and conducting an extensive search of evidence-based sources is an effective way to identify current evidence for use in practice. The clinical question often is developed using the PICO format, which includes the following elements:

P – population or participants of interest in your clinical setting

I – intervention needed for practice

C – comparisons of interventions to determine the best intervention for your practice

O– outcomes needed for practice and ways to measure the outcomes in your practice

The PICO format helps you organize the search for research evidence in a variety of databases and websites. You can identify research syntheses (systematic reviews, meta-analyses, meta-syntheses, and mixed-methods systematic reviews); evidence-based guidelines, protocols, and algorithms; and individual studies through searches of electronic databases, national library sites, and EBP organizations and collections. Some of the key resources for EBP are identified in Table 13.1. At least 2500 new systematic reviews are reported in English and indexed in the Medical Literature Analysis and Retrieval System Online (MEDLINE) each year. The Cochrane Collaboration (2017) library of systematic reviews is an excellent resource, with more than 11,000 entries relevant to nursing and health care. In 2009, the Cochrane Nursing Care (CNC) Field was developed to support the conduct, dissemination, and use of systematic reviews in nursing. The CNC Field produces the Cochrane Corner columns (summaries of Cochrane Reviews relevant to nursing care) that are regularly published in collaborating nursing care − related journals (CNC, 2017). The Joanna Briggs Institute (2017) also provides resources for locating and conducting research syntheses in nursing. The Nursing Reference Center (NRC) includes evidence-based care sheets for numerous nursing interventions and clinical conditions (see Table 13.1).

Table 13.1

Evidence-based practice resources

RESOURCE DESCRIPTION

Electronic Databases

CINAHL (Cumulative Index to Nursing and Allied Health Literature) CINAHL is an authoritative resource covering the English language journal literature for nursing and allied health. The database was developed in the United States and includes sources published from 1982 to the present.

MEDLINE (PubMed, National Library of Medicine) MEDLINE was developed by the National Library of Medicine in the United States; it provides access to more than 11 million MEDLINE citations back to the mid-1960s and to additional life science journals.

MEDLINE with MeSH Also developed by the National Library of Medicine, MEDLINE with MeSH provides authoritative medical information on medicine, nursing, dentistry, veterinary medicine, the healthcare system, preclinical services, and more.

PsycINFO The American Psychological Association developed this database that includes professional and academic literature for psychology and related disciplines from 1887 to the present.

CANCERLIT CANCERLIT, containing information on cancer, was developed by the National Cancer Institute in the United States.

National Library Sites

Cochrane Library The Cochrane Library provides high-quality evidence for those providing and receiving health care and those involved in research, teaching, funding, and administration of health care at all levels. Included is the Cochrane Collaboration, which has many systematic reviews of research (http://www.cochrane.org/evidence).

National Library of Health (NLH) The NLH, located in the United Kingdom, provides searchable evidence-based sources at http://www.evidence.nhs.uk.

Evidence-Based Practice Organizations and Collections

National Guideline Clearinghouse (NGC) The Agency for Healthcare Research and Quality (AHRQ) developed the NGC to house the thousands of evidence-based guidelines that have been developed for use in clinical practice; these can be accessed online at http://www.guidelines.gov.

Cochrane Nursing Care (CNC) Field The Cochrane Collaboration includes over 8000 reviews in 11 different fields; one is the CNC, which supports the conduct, dissemination, and use of systematic reviews in nursing. Most libraries subscribe to the Cochrane Collaboration but free access to abstracts and reviews can be found at http://cncf.cochrane.org.

National Institute for Health and Clinical Excellence (NICE) The NICE was organized in the United Kingdom to provide access to current evidence-based guidelines, similar to the NGC (http://nice.org.uk).

Joanna Briggs Institute (JBI) JBI, an international evidence-based organization originating in Australia, has a search website that includes evidence summaries, systematic reviews, systematic review protocols, evidence-based recommendations for practice, best practice information sheets, consumer information sheets, and technical reports; see “Search the Joanna Briggs Institute” (http://www.joannabriggs.org).

Nursing Reference Center (NRC) The NRC includes a collection of rigorously reviewed, evidence-based care sheets that provide current best practice for over 700 interventions and clinical conditions. This source requires a subscription, so check with your librarian. You can access this resource at http://www.ebscohost.com/nursing.

Evidence Focused on Aspiration During Intramuscular Injections

You might pose a clinical question about whether nurses should aspirate or not when giving intramuscular (IM) injections. Using the PICO format you can identify the evidence needed for practice.

P – populations: infants, toddlers, children, and adults receiving immunizations by the IM route for prophylactic purposes.

I – intervention: IM injection given without aspiration in the right site based on the volume of medication and age of patient (Ogston-Tuck, 2014; Sisson, 2015; Thomas, Mraz, & Rajcan, 2016; Wynaden et al., 2015).

C – comparison intervention: IM injection given with 5 to 10 seconds of aspiration in all sites, regardless of the age of the patient and the volume of medication (Cocoman & Murray, 2008; Nicoll & Hesby, 2002).

O – outcome: IM injection of vaccine without complications.

Older evidence-based guidelines by Nicoll and Hesby (2002) and Cocoman and Murray (2008) recommended aspiration for 5 to 10 seconds with each IM injection to prevent injecting substances directly into a patient’s bloodstream. However, a systematic review by Sisson (2015) recommended no aspiration with IM injections given in the deltoid, ventrogluteal, and vastus lateralis sites. Nurses should only aspirate when giving IM injections in the dorsogluteal site because of the close proximity of the gluteal artery. However, researchers recommended that the dorsogluteal site not be used, if possible (Ogston-Tuck, 2014; Sisson, 2015; Wynaden et al., 2015). The current research evidence regarding aspiration during IM injections is summarized in Box 13.1. However, many nurses are not using this current research evidence about IM injections in practice. Thomas et al. (2016) found that 74% of the nurses were still aspirating after IM injections 90% of the time. Wynaden et al. (2015) found a higher use of the dorsogluteal site, even though current research recommends use of the ventrogluteal site. Therefore these researchers recommended additional education in nursing programs and continuing education that ensure nurses are knowledgeable about and use the most current research evidence in practice.

Box 13.1

Clinical Practice Guideline: Intramuscular Injections Without Aspiration

Patient Population

Infants, toddlers, children, and adults receiving immunizations by the IM route for prophylactic purposes

Objective

Administration of IM immunizations to eliminate patient injury and discomfort

Intervention: IM Injection

Site selection based on the age of the patient

(Nicoll & Hesby, 2002; Ogston-Tuck, 2014; Sisson, 2015; Wynaden et al., 2015):

• Infants – vastus lateralis is the preferred site

• Toddlers and children – vastus lateralis or deltoid sites

• Adults – ventrogluteal or deltoid sites

Medication volume

(Nicoll & Hesby, 2002; Sisson, 2015; Wynaden et al., 2015)

• Small volumes of medication (≤ 2 mL) may be given in the deltoid site for toddlers, children, and adults and in the vastus lateralis for infants.

• Large volumes of medication (2 − 5 mL) should be given in the ventrogluteal site for adults. Volume must be limited and injected in the vastus lateralis for infants, toddlers, and children.

Injection without and with aspiration

•  Cleanse the site with alcohol and allow it to dry.

•  Insert the needle into the appropriate site.

• There should be no aspiration with deltoid, ventrogluteal, and vastus lateralis sites (Sisson, 2015; Thomas et al., 2016; Wynaden et al., 2015).

• Aspirate for 5 to 10 seconds when using the dorsogluteal site because of the proximity to the gluteal artery, but current research recommends not to use this site (Sisson, 2015; Stringer, 2010; Thomas et al., 2016; Wynaden et al., 2015).

•  Inject medication slowly.

•  Withdraw needle slowly; apply gentle pressure with a dry sponge.

Outcome

•  Assess site for complications, immediately and 2 to 4 hours later, if possible.

•  Record the number and type of complications: pain, redness, and/or warmth.

•  Properly and promptly dispose of all equipment.

IM, Intramuscular.

Adapted from Nicoll, L. H., & Hesby, A. (2002). Intramuscular injections: An integrative research review and guideline for evidence-based practice. Applied Nursing Research, 16(2), 149–162; Ogston-Tuck, S. (2014). Intramuscular injection technique: An evidence-based approach. Nursing Standard, 29(4), 52–59; Sisson, H. (2015). Aspirating during the intramuscular injection procedure: A systematic literature review. Journal of Clinical Nursing, 24(17/18) 2368–2375; Stringer, P. M. (2010). Sciatic nerve injury from intramuscular injections: A persistent and global problem. International Journal of Clinical Practice, 64(11), 1573–1579; Thomas, C. M., Mraz, M., & Rajcan, L. (2016). Blood aspiration during IM injection. Clinical Nursing Research, 25(5), 549–559; Wynaden, D., Tohotoa, J., Omari, O. A., Happell, B., Heslop, K., Barr, L., & Sourinathan, V. (2015). Administering intramuscular injections: How does research translate into practice over time in the mental health setting? Nurse Education Today, 35(1), 620–624.

Critically appraising research syntheses: systematic reviews and meta-analyses

Research evidence is usually synthesized using systematic review, meta-analysis, meta-synthesis, and mixed-methods systematic review (Whittemore, Chao, Jang, Minges, & Park, 2014). As noted earlier, Sisson (2015) conducted a systematic review to synthesize research related to IM injections and recommended that nurses not aspirate when giving most IM injections (see Box 13.1). Nursing students and RNs must be able to review research syntheses and determine the evidence to use in practice. This section provides guidelines for understanding and critically appraising systematic reviews and meta-analyses.

Critically Appraising Systematic Reviews

A systematic review is a structured, comprehensive synthesis of the research literature to determine the best research evidence available to address a healthcare question or problem. A systematic review involves identifying, locating, appraising, and synthesizing quality research evidence for clinicians to use in practice (Bettany-Saltikov, 2010a, 2010b; Cooper, 2017; Liberati et al., 2009; Moher, Liberati, Tezlaff, Altman, & PRISMA Group, 2009; Setia, 2016). Systematic reviews are often conducted by two or more researchers and/or expert clinicians in a selected healthcare area to determine the best research knowledge in that area.

Systematic reviews should include rigorous research methodology to promote the accuracy of the findings and minimize the reviewers’ bias. Table 13.2 provides a checklist for critically appraising the steps or elements of systematic reviews and meta-analyses. These steps are based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (Liberati et al., 2009; Moher et al., 2009). The PRISMA statement was developed in 2009 by an international group of expert researchers and clinicians to improve the quality of reporting for systematic reviews and meta-analyses. It includes 27 items, which can be found at http://prisma-statement.org and are detailed in the articles by Liberati et al. (2009) and Moher et al. (2009). These 27 items were consolidated into the checklist in Table 13.2 to assist you in critically appraising systematic reviews and meta-analyses.

Table 13.2

Checklist for critically appraising published systematic reviews and meta-analyses

SYSTEMATIC REVIEW STEPS OR ELEMENTS STEP COMPLETE? (YES OR NO) COMMENTS: QUALITY AND RATIONALE

1.  Did the title indicate that a systematic review, meta-analysis, or both were conducted?

2.  Was an abstract included that provided a structured summary of purpose, data sources, study eligibility criteria, study appraisal and synthesis methods, participants, interventions, outcomes, key findings, conclusions, and/or implications for practice?

3.  Was the clinical question clearly expressed and significant? Was the PICOS format (participants, intervention, comparative interventions, outcomes, and study design) used to develop the question and focus the systematic review or meta-analysis?

4.  Were the purpose and/or objectives or aims of the research synthesis clearly expressed and used to direct it?

5.  Were the search criteria clearly identified? Were the years covered, language, and publication status of sources identified in the search criteria?

6.  Was a comprehensive, systematic search of the literature conducted using explicit criteria identified in step 5? Were the search strategies clearly reported with examples? Did the search include published studies, grey literature, and unpublished studies?

7.  Was the process for selecting studies for the review clearly identified and consistently implemented? Was the selection process expressed in a flow diagram?

8.  Were the publication biases addressed, such as time lag bias, location bias, duplicate publication bias, citation bias, and language bias?

9.  Were key elements (population, sampling process, design, intervention, outcomes, and results) of each study clearly discussed and presented in a table?

10.  Was a quality critical appraisal of the studies conducted? Were the results related to participants, types of interventions, outcomes, and outcome measurement methods clearly presented in tables and narrative? Were the risks for methodological and outcome reporting biases addressed for the studies?

11.  Was a meta-analysis conducted as part of the systematic review? Was a rationale provided for conducting the meta-analysis? Were the details of the meta-analysis process and results clearly described?

12.  Were the results of the systematic review or meta-analysis clearly described (i.e., in a narrative and table)? Were details of the study interventions compared and contrasted in a table? Were the outcome variables clearly identified and the quality of the measurement methods addressed?

13.  Did the report conclude with a clear discussion section?

a. Were the review findings summarized to identify the current best research evidence?

b. Were the limitations of the review and how they might have affected the findings addressed?

c. Were the recommendations for further research and practice addressed?

14.  Did the authors of the review develop a clear, concise, quality report for publication? Was the report inclusive of the items identified in the PRISMA statement in this table?

Adapted from Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P., et al. (2009). The PRISMA Statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: Explanation and elaboration. Annals of Internal Medicine, 151(4), W-65–W-94; and Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & PRISMA Group. (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. http://www.prisma-statement.org.

PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

The systematic review by Holmen, Wahl, Småstuen, and Ribu (2017), which focused on the use of mobile apps for feedback between patients with diabetes and healthcare professionals, is presented as an example. You can find this systematic review online in the Cumulative Index to Nursing and Allied Health Literature (CINAHL) database (see Table 13.1). We recommend that you read this article and use the guidelines in Table 13.2 to critically appraise this systematic review and compare your findings with the following discussion.

Step 1: Did the title indicate if a systematic review or meta-analysis was conducted?

Holmen et al. (2017, e227) identified the type of research synthesis that they conducted in their report title: “Tailored communication within mobile apps for diabetes self-management: A systematic review.”

Step 2: Did the abstract include a structured summary of the research synthesis?

Holmen et al. (2017) provided a clear, concise abstract that was structured by the following headings: background, objective, methods, results, and conclusions of the review.

Step 3: Was a significant, clear clinical question developed to direct the research synthesis?

A systemic review or meta-analysis is best directed by a relevant clinical question that focuses the review process and promotes the development of a quality synthesis of research evidence. The PICOS format is most commonly used to develop clinical questions for research syntheses. The PICOS format (similar to the PICO format introduced earlier) is included in the PRISMA Statement (Moher et al., 2009) with the following elements:

P – population or participants of interest (see Chapter 9 on sampling)

I – intervention needed for practice (see Chapter 8 on nursing interventions)

C – comparisons of the intervention with control, placebo, standard care, variations of the same intervention, or different therapies (see Chapter 8)

O – outcomes needed for practice (see Chapter 10 on measurement methods and Chapter 14 on outcomes research)

S – study design (see Chapter 8 on types of study designs)

Holmen et al. (2017) did not provide a clinical question to direct their systematic review but they did provide a strong background for the review. The researchers reported that “About 415 million people have diabetes globally, and management of diabetes and its complications remains a global health emergency that already accounts for 12% of global health expenditures… The mobile health (mHealth) literature indicates that individuals using mobile apps for self-management achieve positive health outcomes” (Holmen et al., 2017, e227.1). The ideas from the report are summarized using the PICOS format:

P – population: patients diagnosed with diabetes

I – intervention: communication and tailored feedback using mobile apps between patients and healthcare professionals

C – comparisons of the intervention with controls and usual care

O – outcomes examined included hemoglobin A1c (HbA1c), blood pressure, satisfaction with mobile app

S – study designs: quasi-experimental and experimental clinical trials

Step 4: Were the purpose and/or the objectives or aims of the review expressed?

Systematic reviews of research might include a purpose or sometimes specific aims or objectives to guide the synthesis process (Bettany-Saltikov, 2010a; Moher et al., 2009; Setia, 2016). Holmen et al. (2017, e227.1) reported:

To the best of our knowledge, results based on apps with integrated and tailored communication alone have not been systematically summarized. This review aims to address the knowledge gap by systematically reviewing studies that aimed to evaluate integrated communication within mobile apps for tailored feedback between patient with diabetes and HCP [health care providers] in terms of (1) study characteristics, (2) functions, (3) study outcomes, (4) effects, and (5) methodological quality.

Step 5: Was the literature search criteria clearly identified?

Research reports of systematic reviews or meta-analyses need to identify the inclusion and exclusion criteria used to direct the literature search (see Table 13.2). The PICOS format might be used to develop the search criteria with more detail being developed for each of the elements. These search criteria might focus on the following: (1) type of research methods, quantitative, qualitative, or mixed methods; (2) population or type of study participants; (3) study designs, such as quasi-experimental and experimental; (4) sampling processes, such as probability or nonprobability sampling methods; (5) intervention and comparison interventions; and (6) specific outcomes to be measured. The search criteria also need to indicate the years for the review, language, and publication status of the studies to be included (Bettany-Saltikov, 2010b; Higgins & Green, 2008).

Holmen et al. (2017) reported specific eligibility criteria for their literature search. The studies included in the review had to test a mobile app with communication between patients with diabetes and their HCPs. The studies had to include a control or usual care group and have a quasi- experimental or experimental design. Studies that focused on primary prevention of diabetes or that included participants with gestational diabetes were excluded.

Step 6: Was a comprehensive, systematic search of the research literature conducted?

The key search terms, different databases searched, and search results should be recorded in the systematic review and meta-analysis publications. Sometimes authors provide a table that identifies the search terms and criteria. The PRISMA statement recommends presenting the full electronic search strategy used for at least one major database, such as CINAHL or MEDLINE (Liberati et al., 2009).

Often, searches are limited to published sources in common databases, which excludes the grey literature from the research synthesis. Grey literature refers to studies that have limited distributions, such as theses and dissertations, unpublished research reports, articles in obscure journals, articles in some online journals, conference papers and abstracts, conference proceedings, research reports to funding agencies, and technical reports (Conn, Valentine, Cooper, & Rantz, 2003). Most grey literature is difficult to access through database searches and is often not peer-reviewed, with limited referencing information. These are some of the main reasons why grey literature is not included in systematic reviews and meta-analyses. However, excluding grey literature from any type of research synthesis might result in misleading biased results (Pappas & Williams, 2011).

Holmen et al. (2017) detailed their search strategy for their systematic review that included an extensive number of databases and other sources. They provided an example of the search strategy that was applied in MEDLINE. However, there was no mention of including grey literature in the review, which could have biased the findings. The literature search strategy is briefly presented in the following quote:

A systematic literature search was conducted according to the PRISMA guidelines (Moher et al., 2009). Medical literature published from January 2008 was searched in January 2016, with an updated search closed on September 23, 2016, using Medical Literature Analysis and Retrieval System Online (MEDLINE), PubMed, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Excerpta Medica database (EMBASE), ClinicalTrials.gov, and the World Health Organization (WHO) International Clinical Trials Registry Platform. We reviewed reference lists of relevant reviews and studies, and we also conducted hand searches in relevant journals of the field in addition to studies based on tips from colleagues in the field.… we organized a search strategy consisting of the terms mobile applications, cell phones, mobile phones… diabetes mellitus… diabetes mellitus type 2.… We did not set a language limitation; however we did set a limitation on publication year… as we decided technologies prior to 2008 were unlikely to be mobile apps.”

Holmen et al., 2017, e227.3

Step 7: Was the process for selecting the studies for review detailed?

The selection of studies for inclusion in a systematic review or meta-analysis is a complex process that initially involves the review and removal of duplicate sources. The abstracts of the remaining studies are reviewed by two or more authors and sometimes by an external reviewer to ensure that they meet the criteria identified in step 5 (see Table 13.2). The abstracts might be excluded based on the study participants, interventions, outcomes, or design not meeting the search criteria (Bettany-Saltikov, 2010b). The study selection process is best demonstrated by a flow diagram that was developed by the PRISMA Group (Moher et al., 2009). Holmen et al. (2017) provided a detailed description of the process they used to select studies for review. The selection process was documented using a flow diagram that identified the six studies included in the review. The following quote briefly presents the study selection process.

Study Selection

Two reviewers (HH and LR [authors]) independently reviewed all the titles and/or abstracts from the search. We applied our inclusion and exclusion criteria set a priori. For possibly eligible studies, a full text copy was retrieved and reviewed independently by HH and LR. Discrepancies were resolved by discussion or with the involvement of a third reviewer (AKW).…

A total of 2822 papers were identified during the search (Fig. 1). After the removal of 1694 duplicates, the remaining 1128 citations were screened through title and/or abstract, and we removed 913 citations because they clearly did not meet our inclusion criteria. The full text of the remaining 215 citations was then obtained to clarify their study details, and we contacted 22 authors to clarify that their intervention consisted of an app with integrated and tailored communication… After the termination of the search, 6 citations were included in this review.

Holmen et al., 2017, e227.4 − 5

Fig. 1 documents the selection process that was used to determine the studies included in Holmen et al.’s (2017) review. This diagram included the four phases identified by the PRIMA Statement (Moher et al., 2009): (1) identification of the sources; (2) screening of the sources based on set criteria; (3) determining if the sources meet eligibility requirements; and (4) identifying the studies included in the review.

FIG 1 Flowchart. (From Holmen, H., Wahl, A. K., Småstuen, M. C., & Ribu, L. [2017]. Tailored communication within mobile apps for diabetes self-management: A systematic review. Journal of Medical Internet Research, 19[6], e227.)

Step 8: Were publication biases addressed?

Even with rigorous literature searches, authors of meta-analyses and systematic reviews are often limited to mainly published studies. The nature of the sources can lead to biases and flawed or inaccurate conclusions in the research syntheses. The common publication biases that can occur in conducting and reporting research syntheses include time lag bias, location bias, duplicate publication bias, citation bias, and language bias. Publication bias occurs because studies with positive results are more likely to be published than studies with negative or inconclusive results. Higgins and Green (2008) found that the odds were four times greater that positive study results would be published versus negative results. Time lag bias of studies, a type of publication bias, occurs because studies with negative results are usually published later, sometimes 2 to 3 years later, than studies with positive results. Sometimes, studies with negative results are not published at all, whereas studies with positive results might be published more than once (duplicate publication bias). Location bias of studies can occur if studies are published in lower impact journals and indexed in less searched databases. A citation bias occurs when certain studies are cited more often than others and are more likely to be identified in database searches. Language bias can occur if searches focus just on studies in English, and important studies exist in other languages. Holmen et al.’s (2017) systematic review had the potential for publication bias because grey literature was not addressed. However, language was not a limitation of this review, and duplicate abstracts were removed to prevent duplication bias. The time frame of the search was strong based on the topic of mobile apps.

Step 9: Were key elements of the studies presented?

Key elements of studies in systematic reviews and meta-analyses are best identified by constructing a table describing the characteristics of the included studies, such as the purposes of the studies, populations, sampling processes, interventions, outcomes, and results (Bettany-Saltikov, 2010b; Liberati et al., 2009). Holmen et al. (2017) developed three tables to document key information from the six studies they reviewed. One table summarized the study characteristics such as authors, year of publication, country where the study was published, randomization to groups, and attrition of study participants. The studies were from six different countries, and four were RCTs. A second table presented the designs of the studies, and a third table summarized the characteristics of the study participants.

Step 10: Were the studies critically appraised and the risks for biases described?

Two or more experts need to review the studies independently and make judgments about their quality. The critical appraisal of the studies is often difficult because of the differences in types of participants, designs, sampling methods, interventions, outcome variables and measurement methods, and presentation of results. The studies are often rank-ordered based on their quality and contribution to the development of the review (Bettany-Saltikov 2010b; Liberati et al., 2009). Holmen et al. (2017) conducted the Cochrane methodology for risk of systematic bias (ROB) to score each study (Higgins & Green, 2008). The ROB was performed individually by three of the researchers and the scores were discussed to achieve consensus.

Holmen et al. (2017) also provided a detailed discussion and tables to identify the risks for methodological and outcome reporting biases in the studies reviewed. Methodological bias is often related to design and data analysis problems in studies. For example, studies might have limitations related to the sample, intervention, outcome measurements, and analysis techniques that result in methodological bias. The following methodological biases were reported for the six studies reviewed by Holmen et al. (2017): (1) varied mobile apps (interventions) implemented; (2) limited blinding of personnel and participants to group assignment; and (3) insufficiently reported randomization procedures. Outcome reporting bias occurs when study results are not reported clearly and with complete accuracy. For example, reporting bias occurs when researchers selectively report positive results and not negative results, or positive results might be addressed in detail, with limited discussion of negative results. Holmen et al. (2017) reported incomplete outcome data because of attrition in four of the six studies reviewed. Selective reporting of outcomes was noted in five of the studies.

Step 11: Was a meta-analysis conducted as part of the systematic review?

Some authors conduct a meta-analysis in the synthesis of sources for their systematic review (Liberati et al., 2009). Because a meta-analysis involves the use of statistics to summarize results of different studies, it usually provides strong, objective information about the effectiveness of an intervention or solid knowledge about a clinical problem. For example, a meta-analysis might be conducted on a small group of similar studies to determine the effect of an intervention. The systematic review conducted by Holmen et al. (2017) did not include a meta-analysis because the studies lack rigorous methodological quality to be combined in a meta-analysis.

Step 12: Were the results of the review clearly presented?

The results of a systematic review and meta-analysis should include a description of the study participants, types of interventions implemented in the studies, outcomes measured, and measurement methods. The results of the different types of intervention might be best summarized in a table that includes the following: (1) study source; (2) structure of the intervention (stand-alone or multifaceted); (3) specific type of intervention (e.g., physiological treatment, education, counseling, behavioral therapy); (4) delivery method (e.g., demonstration and return demonstration, verbal, video, self-administered); (5) length of time the intervention is implemented; and (6) statistical differences between the intervention and control, standard care, placebo, or alternative intervention groups (Liberati et al., 2009).

The systematic review by Holmen et al. (2017) focused on the intervention of communication between patients and HCPs using mobile apps. The six studies reviewed included mobile apps of various forms and functions, which were summarized in a table. The following comments were made about the apps’ capabilities: “The feedback used was either automatic or manual… and 4 apps also offered direct messages from the patient in free text… A total of 2 apps had critical alerts sent to the patients if their entered readings were outside present thresholds” (Holmen et al., 2017, e227. 6).

The outcomes, including primary and secondary outcomes, of the reviewed studies are best summarized in a table. This table might include: (1) the study source; (2) outcome variable(s), with an indication as to whether it was a primary or secondary outcome in the study; (3) measurement method used for each study outcome variable; and (4) quality of the measurement methods, such as the reliability and validity of a scale or the precision and accuracy of a physiological measure. Holmen et al. (2017) summarized the primary and secondary outcomes from their review in a table and noted that the primary and secondary outcomes varied for the six studies. The HbA1c was a primary outcome in four studies, and only two studies reported a significant decrease in the HbA1c. Blood pressure (BP) was a primary outcome for three studies, but two trials did not measure BP. Secondary outcomes regarding knowledge of diabetes and usability and satisfaction with mobile apps were inconsistently measured and reported, so no conclusions could be drawn.

Step 13: Did the report conclude with a clear discussion section?

In a systematic review or meta-analysis, the discussion of the findings includes an overall evaluation of the types of interventions implemented and the outcomes measured. You can also expect the methodological issues or limitations of the review to be addressed. Finally, the discussion section needs to provide conclusions and recommendations for further research and practice (Bettany-Saltikov, 2010b; Higgins & Green, 2008; Liberati et al., 2009).

Holmen et al. (2017) provided a discussion of their findings, limitations, and recommendations for research and practice. Overall, the methodological problems of these studies provided varied results and limited conclusive findings about the use of mobile apps that are not ready to guide practice.

Conclusion

The conclusions from this systematic review are limited. The unclear and poor methodological quality of this emerging research field is of major concern, and although 3 studies found that apps with integrated feedback significantly improve the primary outcome, the evidence has limitations because of its poor methodological quality. Mobile apps will be a part of the health care system in the future; therefore, we require robust research in this area to make the right choices for the patient, for the health care system, and for society.

Holmen et al., 2017, e227.11 − 12

Step 14: Was a clear and concise report developed for publication?

The systematic review or meta-analysis report needs to include the content discussed in the previous 13 steps. When critically appraising a systematic review, you can use Table 13.2, to indicate if the step is present, and comment about its quality with supporting rationale. In summary, Holmen et al. (2017) developed a quality systematic review following the PRISMA guidelines for publication. The title clearly indicated the type of synthesis conducted. No clinical question was expressed to direct the review but a background was provided so the PICOS format might be developed. A knowledge gap was identified for nursing practice, and the aim of the review focused on this area. The search of the literature might have been more rigorous and might have included additional studies, especially grey literature. The selection of studies for the synthesis was clearly presented in a flow chart and documented with rationale. The studies selected for the systematic review were critically appraised, and the results from these syntheses were clearly presented in tables and narrative. The publication concluded with appropriate findings, limitations, and recommendations for research and practice.

Critically Appraising Meta-Analyses

A meta-analysis is conducted to pool or combine statistically the results from previous studies into a single quantitative analysis that provides one of the highest levels of evidence about the effectiveness of an intervention (Andrel, Keith, & Leiby, 2009; Cooper, 2017; Liberati et al., 2009). This approach has specific objectivity because it includes analysis techniques to determine the effect of an intervention while examining the influences of variations in the studies included in the meta-analysis. Heterogeneity in the studies included in a meta-analysis can lead to different types of methodological and outcome reporting biases previously discussed. Meta-analyses that include more homogeneous (similar) studies have less bias and usually provide more valid findings (Moore, 2012).

Statistically combining data from several studies results in a large sample size, with increased power to determine the true effect of a specific intervention. The ultimate goal of a meta-analysis is to determine if an intervention: (1) significantly improves outcomes; (2) has minimal or no effect on outcomes; or (3) increases the risk of adverse events. Meta-analysis is also an effective way to resolve conflicting study findings and controversies that may have arisen related to a selected intervention (Higgins & Green, 2008).

Strong evidence for using an intervention in practice can be generated from a meta-analysis of multiple quality studies, such as RCTs and other experimental studies. However, the conduct of a meta-analysis depends on the accuracy, clarity, and completeness of information presented in studies. Box 13.2 provides a list of information that needs to be included in a research report to facilitate the conduct of a meta-analysis. You might use this information as a checklist to determine if the reports of RCTs and other interventional studies are complete.

Box 13.2

Recommended Reporting by Researchers to Facilitate the Conduct of Meta-Analyses

Demographic Variables Relevant to Population Studied

•  Age

•  Gender

•  Marital status

•  Ethnicity

•  Education

•  Socioeconomic status

Methodological Characteristics

•  Sample size (experimental and control groups)

•  Type of sampling method

•  Sampling refusal rate and attrition rate

•  Sample characteristics

•  Research design

•  Groups included in study—experimental, control, comparison, placebo groups

•  Intervention protocol and fidelity discussion

•  Data collection techniques

•  Outcome measurements

• Reliability and validity of instruments

• Precision and accuracy of physiological measures

Data Analysis

•  Names of statistical tests

•  Sample size for each statistical test

•  Degrees of freedom for each statistical test

•  Exact value of each statistical test

•  Exact p value for each test statistic

•  One-tailed or two-tailed statistical test

•  Measures of central tendency (mean, median, and mode)

•  Measures of dispersion (range, standard deviation)

•  Post hoc test values for ANOVA (analysis of variance) test of three or more groups

The steps for critically appraising a meta-analysis are similar to those for critically appraising a systematic review (detailed earlier in Table 13.2). The PRISMA statement (Moher et al., 2009), Cochrane Collaboration guidelines for meta-analysis (Higgins & Green, 2008), and other resources (Andrel et al., 2009; Moore, 2012; Setia, 2016; Turlik, 2010) were used in critically appraising a meta-analysis. Conn’s (2010) meta-analysis to determine the effect of physical activity (PA) interventions on depressive symptom outcomes in healthy adults is presented as an example.

Clinical Question Directing a Meta-Analysis

The clinical question developed for a meta-analysis is usually clearly focused: “What is the effectiveness of a selected intervention?” The PICOS (participants or population, intervention, comparative interventions, outcomes, and study design) format discussed earlier might be used to generate the clinical question (Moher et al., 2009). Conn (2010) reported that only one previous meta- analysis had examined the effect of PAs on depressive symptoms among study participants without clinical depression. The meta-analysis conducted by Conn focused on the following clinical question: “What is the effect of PA on depressive symptoms in healthy adults?”

Purpose and Questions to Direct a Meta-Analysis

Researchers need to identify the purpose of their meta-analysis and the questions or objectives that guide the analysis. Conn (2010) clearly identified the following relevant purpose and research questions to guide her meta-analysis:

This meta-analysis synthesized depressive symptom outcomes of supervised and unsupervised PA interventions among healthy adults.… This meta-analysis addressed the following research questions:

(1) What are the overall effects of supervised PA and unsupervised PA interventions on depressive symptoms in healthy adults without clinical depression?

(2) Do interventions’ effects on depressive symptom outcomes vary depending on intervention, sample, and research design characteristics?

(3) What are the effects of interventions on depressive symptoms among studies comparing treatment subjects with before versus after interventions?

Conn, 2010, pp. 128 − 129

Search Criteria and Strategies for Meta-Analyses

The search criteria are usually more narrowly focused for a meta-analysis than a systematic review to identify the specific studies examining the effect of a particular intervention. Conn (2010) clearly identified her detailed search strategies in the following excerpt. She used ancestry searches, which involves the use of citations from relevant studies to identify additional studies.

Primary Study Search Strategies

Multiple search strategies were used to ensure a comprehensive search and thus limit bias while moving beyond previous reviews. An expert reference librarian searched 11 computerized databases (e.g., MEDLINE, PsycINFO, EMBASE) using broad search terms (sample MEDLINE intervention terms: adherence, behavior therapy.… PA terms: exercise, physical activity, physical fitness)…. Search terms for depressive symptoms were not used to narrow the search because many PA intervention studies report depressive symptom outcomes but do not consider these the main outcomes of the study and thus papers are not indexed by these terms…. Computerized author searches were completed for project principal investigators located from research registers and for the first three authors on eligible studies. Author searches were completed for dissertation authors to locate published papers. Ancestry searches were conducted on eligible and review papers. Hand searches were completed for 114 journals which frequently report PA intervention research.

Conn, 2010, p. 129

Possible Biases for Meta-Analyses

Publication, methodological, and outcome reporting biases can weaken the validity of the findings from meta-analyses. An analysis method termed the funnel plot can be used to assess for biases in a group of studies. This discussion of funnel plots is very brief but will hopefully provide you with some understanding of the funnel plot diagrams included in most meta-analyses.

Funnel plots provide graphic representations of possible effect sizes (ESs) for interventions in selected studies (see Chapter 9 for the calculation of ES). The ES, or strength of an intervention in a study, can be calculated by determining the difference between the experimental and control groups for the outcome variable. The mean difference between the experimental and control groups for several studies is easier to determine if the outcome variable is measured by the same scale or instrument in each study. However, the standardized mean difference (SMD) must be calculated in a meta-analysis when the same outcome, such as depression, is measured by different scales or methods. More details are provided on SMD later in this section.

Fig. 13.1 shows a hypothetical funnel plot of the SMDs from 13 studies. The studies with small sample sizes are toward the bottom of the graph, and the studies with larger samples are toward the top. The SMDs from the studies are fairly symmetrical or are equally divided by the line through the middle of the funnel in the graph. A symmetrical funnel plot indicates limited publication bias. Asymmetry of the funnel plot is mainly the result of publication bias but also of methodological bias, outcome reporting bias, heterogeneity in the studies’ sample sizes and interventions, and chance.

FIG 13.1 Funnel plot of standardized mean differences for hypothetical randomized controlled trials with limited bias. RCTs, Randomized controlled trials. (From Gray, J. R., Grove, S. K., & Sutherland, S. [2017]. The practice of nursing research: Appraisal, synthesis, and generation of evidence [8th ed.]. St. Louis, MO: Elsevier.)

Conn (2010) provided a quality discussions of her literature search and the bias risks in her meta- analysis. The following quote includes key content related to the search results and the risks of biases:

Comprehensive searches yielded 70 reports.… The supervised PA [physical activity] two-group comparison included 1,598 subjects. The unsupervised PA two-group comparison included 1,081 subjects. The treatment single-group comparisons included 1,639 supervised PA and 3,420 unsupervised PA subjects.… Most primary studies were published articles (s = 54), and the remainder were dissertations (s = 14), book chapter (s = 1), and conference presentation materials (s = 1; s indicates the number of reports). Publication bias was evident in the funnel plots for supervised and unsupervised PA two-group outcome comparisons and for treatment group, pre- vs. post-intervention supervised PA and unsupervised PA comparisons.

Conn, 2010, p. 131

Results of Meta-Analysis

Many nursing studies examine continuous outcomes or outcomes that are measured by methods that produced interval- or ratio-level data. Physiological measures to examine blood pressure produce ratio-level data. Likert scales, such as the Center for Epidemiologic Studies Depression Scale (CES-D), produce interval-level data (see Chapter 10 for a copy of the CES-D). Therefore blood pressure and depression are continuous outcomes. The effect of an intervention on a continuous outcome in a meta-analysis is determined by the mean difference between two groups. The mean difference is a standard statistic that identifies the absolute difference between two groups. It is an estimate of the amount of change caused by the intervention (e.g., PA) on the outcome (e.g., depression), on average, compared with the control group. The mean difference is reported in a meta-analysis to identify the effect of an intervention but is appropriate only if the outcome is measured by the same scale in all the studies (Higgins & Green, 2008).

A standardized mean difference (SMD), or d, is a summary statistic that is reported in a meta-analysis when the same outcome is measured by different scales or methods. The SMD is also sometimes referred to as the standardized mean effect size. For example, in the meta-analysis by Conn (2010), depression was commonly measured with three different scales—Profile of Mood States, Beck Depression Inventory, and CES-D. Studies that have differences in means in the same proportion to the standard deviations have the same SMD (d), regardless of the scales used to measure the outcome variable. “The differences in the means and standard deviations in the studies are assumed to be the result of the measurement scales and not variability in the outcome” (Higgins & Green, 2008, p. 256).

Conn’s (2010) meta-analysis result identified a standardized mean effect size of 0.372 (moderate ES) between the treatment and control groups for the 38 supervised PA studies and an SMD of 0.522 (strong ES) among the 22 unsupervised PA studies. (Chapter 9 provides values for determining small, moderate, and strong ESs.) This meta-analysis documented that supervised and unsupervised PA reduced symptoms of depression in healthy adults or adults without clinical depression. Therefore a decrease in depression is another important reason for encouraging patients to be involved in structured and unstructured PAs.

Critically appraising meta-syntheses

Qualitative research synthesis is the process and product of systematically reviewing, critically appraising, and formally integrating qualitative studies to determine knowledge for practice (Butler, Hall, & Copnell, 2016; Finfgeld-Connett, 2010). The name for a synthesis of qualitative research and the process for conducting it are continuing to evolve in nursing. Various types of qualitative research syntheses have appeared in the literature, such as meta-synthesis, meta-ethnography, meta-study, meta-narrative, qualitative meta-summary, qualitative meta-analysis, and aggregated analysis (Barnett-Page & Thomas, 2009; Butler et al., 2016; Sandelowski & Barroso, 2007; Tong, Flemming, McInnes, Oliver, & Craig, 2012). Despite the lack of consensus, qualitative researchers recognize the importance of summarizing qualitative studies to determine current knowledge that might be used in practice, to direct further research, or for policy development.

Meta-synthesis seems to be the more common name for the process of synthesizing qualitative studies (Butler et al., 2016; Melnyk & Fineout-Overholt, 2015; Sandelowski & Barroso, 2007; Tong et al., 2012). In this text, a meta-synthesis is defined as the systematic compilation and integration of qualitative study results to expand understanding and develop a unique interpretation of study findings in a selected area. The focus is on interpretation rather than on combining study results, as with quantitative research synthesis. A meta-synthesis involves the breaking down of findings from different studies to discover essential features and then combining these ideas into a unique, transformed whole. Sandelowski and Barroso (2007) have identified meta-summary as a step in conducting meta-synthesis. A meta-summary is the summarizing of findings across qualitative reports to identify knowledge in a selected area.

Tong et al. (2012) developed the Enhancing Transparency in Reporting Synthesis of Qualitative Research (ENTREQ) statement, and Butler et al. (2016) developed guidelines to promote consistency in reporting qualitative syntheses. Merging ideas from different sources, the following questions were developed to guide students and RNs in critically appraising meta-syntheses. A meta-synthesis conducted by Hall, Leach, Brosnan, and Collins (2017) is critically appraised using these questions.

Critical appraisal guidelines

Critically Appraising Meta-Syntheses

A. Introducing and Framing the Meta-Synthesis

1. Did the title of the report identify it as a meta-synthesis?

2. Did the abstract include the background, aim or clinical question addressed, literature search process, methodology for synthesizing the qualitative studies, results, findings, and conclusions?

3. Did the authors clearly identify the aim or objective of their meta-synthesis?

4. Was the meta-synthesis framed to clarify its focus and scope, making it manageable?

B. Searching the Literature and Selecting Sources

5. Did the authors conduct a systematic and comprehensive search for and retrieval of qualitative studies in the target area of the synthesis?

6. Was the process for selecting studies for the meta-synthesis detailed?

C. Critical Appraisal of Studies and Analysis of Data

7. Was the process for critically appraising the studies described?

8. Was the analysis of the qualitative studies’ findings detailed and the results clearly presented?

D. Discussion of Meta-Synthesis Findings

9. Did the authors clearly discuss the interpretation of the findings from the qualitative studies?

10. Were the findings from the meta-synthesis clearly presented, including the themes identified and/or a model or map of the overall findings?

11. Was the meta-synthesis report complete and concise? (Butler et al., 2016; Finfgeld-Connett, 2010; Higgins & Green, 2008; Tong et al., 2012)?

Introducing and Framing the Meta-Synthesis

In the title of their article, Hall and colleagues (2017) identified their synthesis of nurses’ attitudes toward complementary therapies as a systematic review and meta-synthesis. However, only a meta-synthesis of qualitative studies was included in this article. The researchers provided a quality abstract of their synthesis that included background, aim of the meta-synthesis, search of the literature, ENTREQ process for synthesizing studies, results, discussion of findings, and conclusions.

A meta-synthesis needs to be framed by a clearly stated objective or aim and scope. The aim of the meta-synthesis is usually an important area of interest for the individuals conducting it and is a topic with an adequate body of qualitative studies. The scope of a meta-synthesis is an area of debate, with some qualitative researchers recommending a narrow, precise approach and others recommending a broader, more inclusive approach. However, researchers recognize that framing is essential for making the synthesis process manageable and the findings meaningful and potentially transferable to practice (Butler et al., 2016; Walsh & Downe, 2005).

Hall et al. (2017, p. 48) reported that, “The aim of this meta-synthesis is to review, critically appraise, and synthesize the research to develop a new, more substantial interpretation of nurses’ attitudes regarding complementary therapies.… This review is reported according to the Enhancing Transparency in Reporting the Synthesis of Qualitative Research (ENTREQ) guidelines (Tong et al., 2012).”

Searching the Literature and Selecting Sources

Most authors agree that a rigorous search of the literature needs to be conducted. The search needs to include databases, books and book chapters, full reports of theses and dissertations, and conference reports. Researchers often document the specific search strategies they use to locate relevant qualitative studies for their synthesis. The search criteria need to be detailed in the synthesis report, and the years of the search, keywords searched, and language of the sources need to be discussed. Meta-syntheses are usually limited to qualitative studies only and do not include mixed methods studies (Butler et al., 2016; Whittemore et al., 2014). Also, qualitative findings that have not been analyzed or interpreted, such as unanalyzed quotes, field notes, case histories, stories, and poems, are usually excluded (Finfgeld-Connett, 2010). The search process is usually very fluid, with the conduct of additional computerized and hand searches used to identify more studies. Hall et al. (2017) included the following discussion of their search criteria, search strategies, and the selection of studies for synthesis:

Inclusion and exclusion criteria

Published qualitative empirical studies reporting on nurses’ attitudes toward complementary therapies were included in this review. Nurses could be employed at any level, in any clinical setting, in any country. For this review, we considered complementary therapies as a broad umbrella term rather than focusing on specific products or practices. Studies involving multiple professional groups were excluded due to the potential difficulty in extracting and interpreting data specific to nursing.

Search strategy

A comprehensive search of relevant articles published in English between January 2000 and December 2015 was conducted using the following electronic databases; MEDLINE, CINAHL, and AMED (Allied and Complementary Medicine Database). A list of the terms used in the search is presented in Table 1. Reference lists of included articles were also hand searched for suitable publications, and opengrey.edu, greylit.com and Google Scholar interrogated for pertinent grey literature.

Hall et al., 2017, p. 48

Hall and colleagues (2017) identified appropriate search criteria and clearly implemented them in their report. The search strategies were detailed, and the inclusion of grey literature reduced the potential for publication bias. The search did have a language bias because only studies in English were reviewed. The key search terms were clearly identified in a table (see Table 1), and the search was strengthened by reviewing the sources from the last 15 years.

Table 1

Search Terms

Population Context Outcome

Nurse Complementary medicine

Complementary therapy

Alternative medicine

Alternative therapy

Natural medicine

Natural therapy

Herb

Mind body

Acupuncture Attitude

Perception

Decision making

Behavior

Communication

Experiences

Beliefs

From Hall, H., Leach, M., Brosnan, C., & Collins, M. (2017). Nurses’ attitudes towards complementary therapies: A systematic review and meta-synthesis. International Journal of Nursing Studies, 69(1), 48.

The final selection of studies to include in a meta-synthesis depends on the focus and scope of the synthesis. Some authors focus on one type of qualitative research, such as ethnography, or on one investigator in a particular area. Others include studies with different qualitative methodologies and investigators in a field or related fields. The search criteria need to be consistently implemented in determining the studies to be included and excluded in the synthesis. Hall et al. (2017) included a detailed flow diagram (see Fig. 1) to document their process for selecting the 15 studies included in their meta-synthesis.

FIG 1 Study selection flow diagram. (From Hall, H., Leach, M., Brosnan, C., & Collins, M. [2017]. Nurses’ attitudes towards complementary therapies: A systematic review and meta-synthesis. International Journal of Nursing Studies, 69[1], 49.)

Critical Appraisal of Studies and Analysis of Data

The critical appraisal process for qualitative research varies among sources. Usually a table is developed as part of the appraisal process, but this is also an area of debate because tables of studies are more often included in syntheses of quantitative studies. The table headings might include: (1) authors and year of source; (2) aim or goal of the study; (3) methodological orientation; (4) participants; (5) findings; and (6) other key content relevant for comparison. This table provides a display of relevant study elements so that a comparative appraisal might be conducted (Butler et al., 2016; Finfgeld-Connett, 2010). The comparative analysis of studies involves examining methodology and findings across studies for similarities and differences. The frequency of similar findings might be recorded. The differences or contradictions in studies need to be resolved, explained, or both. Varied analysis techniques often are used by the researchers to translate the findings of the different studies into a new or unique description. Tong et al. (2012) present a summary table of the common methodologies used for the synthesis of qualitative studies.

Hall et al. (2017, p. 49) reported that the “findings of this review are based on the five papers from the USA, three papers from Australia, two from both the UK [United Kingdom] and Thailand, and one paper each from Taiwan, Canada, and Israel.” These 15 studies were presented in detail in a comparative analysis table that included appropriate headings of authors, year, aim, method, participants, and findings. Two reviewers (HH and MC) independently extracted data from the studies. “Data were analyzed using the thematic synthesis process… In the final stage, the reviewers completed an inductive analysis of the themes to develop an interpretation that went beyond the content of original studies” (Hall et al., 2017, p. 49).

Discussion of Meta-Synthesis Findings

A meta-synthesis report includes findings presented in different formats based on the knowledge developed and the perspective of the authors. A synthesis of qualitative studies in one area might result in the discovery of unique or more refined themes explaining the area of synthesis. The findings from a meta-synthesis might be presented in narrative format or graphically presented in a model or map. Authors should also identify the limitations of the meta-synthesis. The report often concludes with recommendations for further research and implications for practice (Butler et al., 2016; Tong et al., 2012; Walsh & Downe, 2005).

Hall et al. (2017, p. 49) identified the following findings: “Five analytical themes relating to nurses’ attitudes towards complementary therapies emerged from the data: the strengths and weaknesses of conventional medicine; complementary therapies as a way to enhance nursing practice; patient empowerment and patient-centeredness; cultural barriers and enablers to integration; and structural barriers and enablers to integration.” The meta-analysis report included a discussion of limitations, implications for practice, and conclusions. Hall et al. (2017, p. 47) concluded that “The nursing profession needs to consider how to address current deficiencies in meeting the growing use of complementary therapies by patients.”

Critically appraising mixed-methods systematic reviews

In recent years, nurse researchers have been conducting mixed methods studies that include quantitative and qualitative research methods (Creswell, 2014; Gray et al., 2017) (see Chapter 14 on mixed methods research). Researchers recognize the importance of synthesizing the findings of these studies to determine important knowledge for practice and future research. Creswell (2014) identified the process of combining findings from quantitative and qualitative studies as mixed methods synthesis. Higgins and Green (2008) referred to this synthesis of quantitative, qualitative, and mixed methods studies as a mixed-methods systematic reviews, which is the term we use in this text.

The systematic reviews discussed earlier in this chapter included only studies of a quantitative methodology, such as meta-analyses, RCTs, and quasi-experimental studies, to determine the effectiveness of an intervention. Mixed-methods systematic reviews might include various study designs, such as different types of qualitative research; descriptive, correlational, and quasi-experimental quantitative studies; and/or mixed methods studies (Bettany-Saltikov, 2010b; Higgins & Green, 2008; Liberati et al., 2009; Whittemore et al., 2014).

Higgins and Green (2008) described two types of approaches to integrate the findings from quantitative, qualitative, and mixed methods studies, multilevel synthesis and parallel synthesis. Multilevel synthesis involves synthesizing the findings from quantitative studies separately from qualitative studies and integrating the findings from these two syntheses in the final report. Parallel synthesis involves the separate synthesis of quantitative and qualitative studies, but the findings from the qualitative synthesis are used in interpreting the synthesized quantitative studies.

Further work is needed to develop the methodology for conducting a mixed-methods systematic review. The steps overlap with those of the systematic review and meta-synthesis processes described earlier. The process might be implemented best with a team of researchers with expertise in conducting different types of studies and research syntheses. Guidelines for critically appraising mixed-methods systematic reviews are presented as follows.

Critical appraisal guidelines

Critically Appraising Mixed-Methods Systematic Reviews

A. Introduction of the Mixed-Methods Systematic Review

1. Did the title identify the type of research synthesis that was conducted?

2. Was a clear, concise abstract presented that included the aim of the review, data sources, study selection process, results, findings, and conclusions?

3. Did the aim and/or questions guide the mixed-methods systematic review?

B. Literature Search Methods and Selection of Sources

4. What were the search criteria for identifying quantitative, qualitative, and mixed methods studies?

5. Were the search strategies detailed enough to identify relevant quantitative, qualitative, and mixed methods studies?

6. Was a rigorous search of the literature conducted and detailed in the final report?

7. Was the process for selecting relevant quantitative, qualitative, and mixed methods studies for the synthesis described?

C. Critical Appraisal of Studies and Results

8. Did the authors of the review present a table and narrative that demonstrated a comparative appraisal of the studies was conducted?

9. Were critical appraisals of the studies summarized in the final report, and were the results provided?

D. Findings, Conclusions, and Implications for Research and Practice

10. Was a clear synthesis of study findings presented? Did this synthesis effectively integrate the findings from quantitative, qualitative, and mixed methods studies?

11. Were the implications for research and practice identified and appropriate (Bettany-Saltikov, 2010a, 2010b; Creswell, 2014; Higgins & Green, 2008)?

Introduction of the Mixed-Methods Systematic Review

Karimi and Clark (2016) conducted a mixed-methods systematic review to determine how patients’ values influenced the self-care decision-making of patients with heart failure (HF). In their title, they stated that a mixed-methods systematic review was conducted. The title also included the clinical question guiding the review: “How do patients’ values influence heart failure self-care decision-making?” (Karimi & Clark, 2016, p. 89). The abstract clearly and concisely covered the background supporting the review, search process, type and number of studies selected for the review, findings, and conclusions. The three objectives or aims of this synthesis were identified early in the review and directed the review process.

Literature Search Methods and Selection of Sources

Karimi and Clark (2016) conducted a comprehensive search of multiple databases and other sources with the assistance of a health sciences librarian. “The search included terms related to three concepts: self-care, values, and HF… The search identified 6467 studies. These were initially screened via title and abstract, resulting in the full-text review of 579 papers (Fig. 1). Of these, 54 met the criteria for inclusion” (Karimi & Clark, 2016, p. 92). As indicated in Fig. 1, the 54 studies reviewed included 30 qualitative studies, 8 mixed methods studies, and 16 quantitative studies. The participants in these studies were 6045 patients, 38 lay caregivers, and 96 healthcare professionals. This mixed-methods systematic review included a strong sample of studies representing the three types of methodologies and included extensive data from numerous patients, caregivers, and health professionals.

FIG 1 Flow chart of literature review. HF, Heart failure. (From Karimi, M., & Clark, A. M. [2016]. How do patients’ values influence heart failure self-care decision-making? A mixed-methods systematic review. International Journal of Nursing Studies, 59[1], 93.)

Critical Appraisal of Studies and Results

The data from the 54 studies were presented in a comparative table format. The headers in the table included author and year, primary focus, methods, country, data collection methods, sampling, participants, New York Heart Association (NYHA) class, mean age, and a study quality score. A comparative analysis was conducted using the table data that was synthesized in the report. Karimi and Clark (2016, pp. 101 − 102) recognized the potential for publication bias because “our search was limited to only English language, peer-reviewed journals or dissertations published from the year 2000 and onwards…. The findings were also constrained given the majority of the studies included were conducted in high income countries. Few described the characteristics of those who declined to participate in the study.” A serious concern was the lack of studies that explicitly addressed patient values, and the authors had to abstract values’ content from the theoretical discussions.

Findings, Conclusions, and Implications for Research and Practice

Karimi and Clark (2016) synthesized their findings into a model of the value-laden, self-care, decision-making process (Fig. 3). The relationships in this model provide direction for practice and should be tested through further research. The authors developed conclusions, implications for practice, and directions for further research based on the findings from their mixed-methods systematic review. They reported the following:

Values are integral to how patients approach and undertake HF self-care… Values extend to those relating to the self and others and incorporate a notable range of personal, life, and social dimensions [see Fig. 3]. Values cannot be assumed to be fixed, normative or similar to those held by nurses and other health professionals. Future interventions to improve HF self-care must address and respond to the complexity of these values and how they influence patient behavior.

Karimi & Clark, 2016, p. 102

FIG 3 The value-laden, self-care, decision-making process. (From Karimi, M., & Clark, A. M. [2016]. How do patients’ values influence heart failure self-care decision-making? A mixed-methods systematic review. International Journal of Nursing Studies, 59[1], 99.)

Models to promote evidence-based practice in nursing

EBP is a complex phenomenon that requires integration of the best research evidence with clinical expertise, patient circumstances, and patient values in the delivery of quality, safe, cost-effective care. The two models most commonly used in nursing to implement research evidence in practice are the Stetler Model of Research Utilization to Facilitate Evidence-Based Practice (Stetler, 2001) and the Iowa Model of Evidence-Based Practice (Iowa Model Collaborative, 2017). These two models are briefly introduced in this section.

Stetler Model of Research Utilization to Facilitate Evidence-Based Practice

The Stetler Model of Research Utilization to Facilitate Evidence-Based Practice (Fig. 13.2) provides a comprehensive framework to enhance the use of research evidence in nursing practice. The research evidence can be used at the institutional or individual level. At the institutional level, study findings are synthesized and the knowledge generated is used to develop or refine policies, algorithms, procedures, protocols, or other formal programs implemented in the institution. Individual nurses, such as RNs, educators, and policy makers, summarize research and use the knowledge to make practice decisions, influence educational programs, and guide political decision making. For example, the evidence-based guideline in Box 13.1 provides knowledge about the age, sites, volume of medication, and aspiration with IM injections that should be implemented by individual nurses to promote quality patient outcomes.

FIG 13.2 Stetler model, part I. Shown are the steps of research utilization to facilitate evidence-based practice. (From Stetler, C. B. [2001]. Updating the Stetler model of research utilization to facilitate evidence-based practice. Nursing Outlook, 49[6], 276.)

Stetler’s model is included in this text to encourage the use of research evidence by individual nurses and healthcare agencies to facilitate the development of evidence-based nursing practice. The five phases of the Stetler (2001) model are briefly described in the following sections: (1) preparation, (2) validation, (3) comparative evaluation and decision making, (4) translation and application, and (5) evaluation.

Phase I: Preparation

The intent of Stetler’s (2001) model is to make using research evidence in practice a conscious, critical thinking process that is initiated by the user. Thus Phase I: Preparation, involves determining the purpose, focus, and potential outcomes of making an evidence-based change in a clinical agency. Once the agency, individuals, or committee have identified and approved the purpose of the evidence-based project, a detailed search of the literature is conducted to determine the strength of the evidence available for use in practice.

Phase II: Validation

In Phase II: Validation, the research reports are critically appraised to determine their scientific soundness (Gray et al., 2017; Grove & Cipher, 2017; Melnyk & Fineout-Overholt, 2015). If the studies are limited in number, weak, or both, the findings and conclusions are considered inadequate for use in practice, and the process stops. If a systematic review, meta-analysis, and/or meta- synthesis has been conducted in the area in which you want to make an evidence-based change, this greatly strengthens the quality of the research evidence. If the research knowledge base is strong in the selected area, the clinical agency or individual nurse must make a decision about using the evidence in practice. For example, the guideline related to IM injections (see Box 13.1) can be implemented by individual nurses to improve their practice outcomes.

Phase III: Comparative Evaluation/Decision Making

Phase III: Comparative Evaluation/Decision Making includes four parts: (1) substantiation of the evidence; (2) fit of the evidence with the healthcare setting; (3) feasibility of using research findings; and (4) concerns with current practice (see Fig. 13.2). Substantiating evidence is produced by systematic reviews and meta-analyses of relevant studies. However, individual quasi-experimental and experimental studies also can provide extremely strong evidence for making a change in an agency. To determine the fit of the evidence in the clinical agency, examine the characteristics of the setting to determine the forces that will facilitate or inhibit implementation of the evidence-based change. Stetler (2001) has noted that the feasibility of using research evidence in practice involves examining the three Rs related to making changes in practice: (1) potential risks; (2) resources needed; and (3) readiness of those involved. By conducting phase III, you can assess the overall benefits and risks of using the research evidence in a practice setting. If the benefits are much greater than the risks for the organization, individual nurse, or both, using the research-based intervention in practice is feasible.

During the decision-making aspect of phase III, three decisions are possible: (1) to use the research evidence; (2) to consider using the evidence; and (3) not to use the research evidence (see Fig. 13.2). The decision to use research knowledge in practice depends mainly on the strength of the evidence. Another decision might be to consider use of the available research evidence in practice. When a change is complex and involves multiple disciplines, additional time is often needed to determine how the evidence might be used and what measures will be taken to coordinate the involvement of different healthcare professionals in the change. A final option might be not to use the research evidence in practice because the current evidence is not strong, or the risks or costs of change in current practice are too high in comparison with the benefits (Stetler, 2001).

Phase IV: Translation/Application

Phase IV: Translation/Application involves planning for and actually using the research evidence in practice. The translation phase involves determining exactly what knowledge will be used and how that knowledge will be applied to practice. The use of the research evidence can be cognitive, instrumental, or symbolic. With cognitive application, the research base is a means of modifying a way of thinking or one’s appreciation of an issue (Stetler, 2001). For example, cognitive application may improve the nurse’s understanding of a situation, allow analysis of practice dynamics, or improve problem-solving skills for clinical problems. Instrumental application involves using research evidence to support the need for change in nursing interventions or practice protocols. Symbolic or political utilization occurs when information is used to support or change a current policy. The application phase includes the following steps for planned change: (1) assess the situation to be changed; (2) develop a plan for change; and (3) implement the plan. During the application phase, the protocols, policies, and/or algorithms developed with research knowledge are implemented in practice.

Phase V: Evaluation

The final stage, Phase V: Evaluation, is to determine the impact of the research-based change on the healthcare agency, personnel, and patients. The evaluation process can include formal and informal activities conducted by administrators, nurse clinicians, and other health professionals. Informal evaluations might include self-monitoring or discussions with patients, families, peers, and other professionals. Formal evaluations can include case studies, audits, QI projects, and translational or outcomes research projects. In summary, the Stetler (2001) model provides detailed steps to encourage nurses to become change agents to make the necessary improvements in practice based on research evidence.

Iowa Model of Evidence-Based Practice

Nurses have been actively involved in conducting research, synthesizing research evidence, and developing evidence-based guidelines for practice. These activities support their strong commitment to EBP, which has been facilitated by the Iowa Model. The Iowa Model of Evidence-Based Practice promotes the implementation of EBP by nurses in clinical agencies. Recently, the Iowa Model Collaborative (2017) conducted a study to revise and validate the Iowa Model. The most current Iowa Model is presented in Fig. 13.3. In healthcare agencies, there are triggering issues or opportunities that initiate the need for change, with the focus on making changes based on the current, best research evidence. The triggering issues are often problem-focused and evolve from clinical problems and data from risk management, process improvement, benchmarking, and financial reports. The opportunities can be knowledge-focused, such as new research findings, changes in national agencies or organizational standards and guidelines, an expanded philosophy of care, an updated professional model of care (requirement of Magnet recognition), or questions from the institutional standards’ committee. The topics identified are evaluated and prioritized based on the needs of the clinical agency. If a topic is considered an agency priority, a group is formed to search for the best evidence to manage the clinical concern (see Fig. 13.3).

FIG 13.3 The Iowa Model Revised: Evidence-Based Practice to Promote Excellence in Health Care. (From Iowa Model Collaborative. [2017]. Iowa Model of Evidence-Based Practice: Revisions and validation. Worldviews on Evidence-Based Nursing, 14[3], 175 − 182, 178.)

In some situations, the research evidence is inadequate to make changes in practice, and additional studies are needed to strengthen the knowledge base (see Fig. 13.3). Sometimes the research evidence can be combined with other sources of knowledge (e.g., theories, scientific principles, expert opinion, case reports) to provide fairly strong evidence for use in developing research-based protocols for practice. Research-based protocols are structured guidelines for implementing nursing interventions in practice that are based on current research evidence. The strongest evidence comes from systematic reviews that include meta-analyses of RCTs. However, meta-syntheses, mixed-methods systematic reviews, and individual studies also provide important evidence for changing practice. The levels of research evidence are described in Chapter 1 (see Fig. 1.3) and are also presented inside the front cover of this text.

The research-based protocols, algorithms, guidelines, or policies developed could be pilot-tested and then evaluated to determine the impact on patient care, personnel, and healthcare agency. If the outcomes are favorable from the pilot test, the change would be appropriate for adoption in practice. Changes would be made by administrators and personnel in the healthcare agency to integrate and sustain the practice change. Key outcomes would be monitored through QI to ensure that the practice change promoted quality, safe, and cost-effective care. The Iowa Model-Revised (see Fig. 13.3) “remains an application-oriented guide for the EBP process” and is a valuable source for implementing research evidence in clinical agencies (Iowa Model Collaborative, 2017, p. 175).

Application of the Iowa Model of Evidence-Based Practice

Preparing to use research evidence in practice raises some important questions:

• Which research findings are ready for use in clinical practice?

• What are the most effective strategies for implementing research-based protocols or evidence-based guidelines in a clinical agency?

• What are the outcomes from using the research evidence in practice?

• Do the risk management data, QI data, benchmarking data, or financial data support making the change in practice based on the research evidence?

• Is the research-based change proposed an agency priority?

We suggest that effective strategies for using research evidence in practice will require a multifaceted approach that takes into consideration the evidence available, attitudes of the practicing nurses, the organization’s philosophy, and national organizational standards and guidelines (ANCC, 2017; Melnyk & Fineout-Overholt, 2015; The Joint Commission, 2017). In this section, the steps of the Iowa Model (see Fig. 13.3) guide the use of a research-based intervention in a hospital to facilitate EBP.

Schumacher, Askew, and Otten (2013) used the Iowa Model to implement a pressure ulcer (PU) trigger tool for assessing the neonatal population in their hospital. PU prevalence ranged from 0% to 1% per quarter in this large Midwest neonatal intensive care unit (NICU). Schumacher and colleagues noted that no PU risk assessment tool was used consistently in their NICU. A summary of the application of the Iowa Model to this clinical problem is presented in Table 1 (Schumacher et al., 2013). The clinical question addressed was generated using the PICO format. The population was neonates in the NICU. The intervention to be implemented was a three-question PU trigger tool to assess if the neonates were at risk for skin breakdown. It was compared to the Braden Q tool, the standard care or usual practice of determining the neonates who were at risk for PUs. The outcome measured was the effectiveness of the PU trigger tool as compared to the Braden Q tool in assessing neonates who were at risk for PU. The studies that had examined the effectiveness of different neonatal PU assessment tools were critically appraised. The tools in these studies were judged to be too long for routine clinical use. Schumacher and colleagues (2013) developed a shorter trigger assessment tool based on the research evidence:

Trigger Questions for Pressure Ulcer Risk Proposed by the Institute for Clinical Systems Improvement: Is the infant: Moving extremities and/or body appropriately for developmental age? Responding to discomfort in a developmentally appropriate manner? Demonstrating adequate tissue perfusion based on the clinical formula (mean arterial pressure = gestational age and/or capillary refill <3 s)?

Schumacher et al., 2013, p. 48

Table 1

Use of the Iowa Model for Evidence-Based Practice Projects to Establish a Clinical Tool for Evaluation of Pressure Ulcer Risk in an NICU

1. Generate the question from either a problem or new knowledge. For infants in the NICU, does the use of a pressure ulcer trigger tool perform equally as well as the Braden Q to identify an infant at risk?

P—Infant in the NICU

I—Use of a pressure ulcer trigger tool

C—Usual practices of assessing all with Braden Q

O—Trigger tool performs equally as well as the Braden Q for risk identification

2. Determine relevance to organizational priorities. Our hospital is committed to safe and reliable care and ensuring a flawless patient experience. This includes preventing hospital-acquired pressure ulcers—a “never event.” Possible impact/outcomes may include:

• Potential increase in WOC referrals

• Correct triggering of infants requiring further pressure ulcer risk assessment and prevention strategies by nursing

• Preservation of nursing time because the trigger tool is a shorter and easier to use tool that indicates for whom full assessment is needed.

3. Develop a team to gather and appraise evidence. Team members included a clinical nurse specialist, two WOC nurses, NICU Nursing Practice Council members, and electronic medical records experts.

According to IHI, all premature infants are at risk for pressure ulcer development. Although this statement is visionary, it does not assist the bedside nurse to determine for whom to provide interventions. Referrals to the WOC nurse for assessment were based on clinical judgment, and no assessment tools were in place. Risk assessment tools are available for the neonatal population but were judged to be lengthy and time-consuming for every nurse, every shift. A trigger tool was developed, based on IHI pediatric trigger questions to help the nurse determine whom to refer for full assessment.

4. Determine if the evidence answers the question. Reasonable evidence is present to warrant implementation of this practice.

The three IHI trigger questions are based on the concepts of the Braden Q and could trigger those at risk and requiring further assessment and intervention.

5. If there is sufficient evidence, pilot the change in practice. Prior to implementation, 10 patients were randomly selected to test the feasibility of implementing trigger questions. The three trigger questions were asked of the nurses caring for each patient, and results were compared to a Braden Q score generated by a WOC nurse. We noted that patients with high risk, as defined by the Braden Q, were also identified as at-risk based on the three trigger questions. The three trigger questions were embedded into the electronic medical record.

Pressure ulcer data were collected through the surveillance provided by skin team nurses as part of their usual duties.

6. Evaluate structure, process, and outcome data. The HAPU rate for the NICU remains low and has not changed since the implementation of the trigger tool. The number of WOC nurse consultations has remained stable since implementation of the three trigger questions into NICU nurse practice.

7. Disseminate results. Results and appropriate feedback have been shared with the NICU nurses via the NICU practice committee.

From Schumacher, B., Askew, M., & Otten, K. (2013). Development of a pressure ulcer trigger tool for the neonatal population. Journal of Wound, Ostomy, and Continence Nursing, 40(1), 47.

HAPU, Hospital-acquired pressure ulcer; IHI, Institute for Healthcare Improvement; NICU, neonatal intensive care unit; WOC, wound, ostomy, and continence.

Schumacher et al. (2013) found that their three-question PU trigger assessment tool was as effective as the Braden Q tool in determining neonatal risk for PUs. Their findings are summarized in the following study excerpt:

Following implementation of the trigger questions in 2009, we observed no net increase in the number of WOC [wound, ostomy, and continence] referrals per 1000 patients. Nevertheless, our PU [pressure ulcer] prevalence in the NICU remains low at 0.01 per 1000 patient days. Comparison of results from the 3 trigger questions and the Braden Q scoring by the WOC nurse demonstrated that most infants are correctly identified by the tool, with the exception of those very immature infants who may be at risk for medical device-related ulcers….

We implemented a 3-item trigger tool to aid NICU nurses identifying neonates at risk for PU development. While these questions do not quantify risk, we have found that they are an efficient initial screening tool when combined with additional assessment and management in consultation with a WOC nurse.

Schumacher et al., 2013, p. 50

Implementing evidence-based guidelines in practice

Research knowledge is generated every day and is synthesized to determine the best evidence for use in practice (Cochrane, 2017; Mackey & Bassendowski, 2017: NGC, 2017a). This section addresses the evidence-based guidelines, based on current best research evidence, published on national and international websites and in referred professional journals. The Eighth Joint National Committee (JNC 8) evidence-based guideline for the management of high blood pressure in adults is presented as an example (James et al., 2014).

Resources for Evidence-Based Guidelines

Since the 1980s, the Agency for Healthcare Research and Quality (AHRQ) has had a major role in identifying health topics and promoting the development of evidence-based guidelines for these topics (http://www.ahrq.gov). The first evidence-based guidelines were developed by panels of nationally recognized researchers in the topic area, expert clinicians (e.g., physicians, nurses, pharmacists, social workers), healthcare administrators, policy developers, economists, government representatives, and consumers. The panel members designated the scope of the guideline and conducted extensive reviews of the literature, including relevant systematic reviews, meta- analyses, individual studies, and theories. The evidence-based guidelines developed were examined by consultants, other researchers, and additional expert clinicians for their input. Based on the experts’ critique, the AHRQ revised and packaged the guidelines for distribution to healthcare professionals.

At present, standardized guideline development ranges from a structured process such as the one just discussed to a less structured process, in which a guideline might be developed by a healthcare organization, healthcare plan, or professional organization. The AHRQ initiated the NGC (NGC, 2017a) in 1998 to store the EBP guidelines. The NGC (2017b) is a publicly available database of evidence-based clinical guidelines and related documents. The NGC is updated weekly with new content that the AHRQ produces in partnership with the American Medical Association and America’s Health Insurance Plans. The key components of the NGC and its user-friendly resources can be found on the AHRQ website (http://www.guideline.gov/). Key information provided by the NGC that is relevant to you is as follows:

• Structured abstracts (summaries) about the guideline and its development

• Links to full-text guidelines, where available, and/or ordering information for print copies

• Downloads of the complete NGC summary for all guidelines represented in the database

• Annotated bibliography database in which users can search for citations for publications and resources about guidelines

• What’s New—enables users to see what guidelines have been added each week; includes an index of all guidelines in the NGC

• The Glossary—provides definitions of terms used in the standardized abstracts (summaries)

The NGC provides an easy to use mechanism for obtaining objective and detailed information on clinical practice guidelines. In addition to the evidence-based guidelines, the AHRQ has developed many tools to assess the quality of care provided by the evidence-based guidelines. You can search the AHRQ (2017a) website (http://www.qualitymeasures.ahrq.gov) for an appropriate tool to measure a variable in a research project or evaluate outcomes of care in a clinical agency. Numerous professional organizations, healthcare agencies, universities, and other groups provide evidence syntheses, evidence-based guidelines for practice, and measurement methods for outcomes, which can be found on the websites listed in Box 13.3.

Box 13.3

Evidence-Based Practice Websites

•  Academic Center for Evidence-Based Nursing: http://www.acestar.uthscsa.edu

•  Association of Women’s Health, Obstetric, and Neonatal Nurses: http://awhonn.org

•  Centers for Disease Control Healthcare Providers: http://www.cdc.gov/CDCForYou/healthcare_providers.html

•  Centers for Health Evidence: http://www.cche.net

•  Guidelines Advisory Committee: http://www.gacguidelines.ca

•  Guidelines International Network: http://www.g-i-n.net/

•  HerbMed: Evidence-Based Herbal Database, 1998, Alternative Medicine Foundation: http://www.herbmed.org/

•  National Association of Neonatal Nurses: http://www.nann.org/

•  National Institute for Clinical Excellence (NICE): http://www.nice.org.uk/

•  Oncology Nursing Society: http://www.ons.org/

•  Primary Care Clinical Practice Guidelines: http://www.medscape.com/pages/editorial/public/pguidelines/index-primarycare

•  US Preventive Services Task Force: http://www.uspreventiveservicestaskforce.org

Implementing an Evidence-Based Guideline for Management of Hypertension in Adults

Evidence-based guidelines have become the standards for providing care to patients in the United States and other countries. The 2014 evidence-based guideline for the management of high blood pressure in adults was developed by the JNC 8 panel members, who conducted a systematic review of RCTs to determine the best research evidence. The guideline includes nine revised recommendations for the management of hypertension (HTN) and are available in the article by James et al. (2014, p. 511) or through the NGC (2017a) Guideline Summary NGC-10397. The JNC 8 guideline also includes the 2014 Hypertension Guideline Management Algorithm. This algorithm provides clinicians with directions for: (1) implementing lifestyle interventions; (2) setting BP goals; and (3) initiating BP-lowering medication based on age, diabetes, and chronic kidney disease (CKD; James et al., 2014). Nurses and other healthcare providers can use this algorithm to select the most appropriate treatment methods for each individual patient diagnosed with HTN.

Nursing students and RNs need to assess the usefulness and quality of each evidence-based guideline before they implement it in their practice. Fig. 13.4 presents the Grove Model for Implementing Evidence-Based Guidelines in Practice. In this model, nurses identify a practice problem, search for the best research evidence to manage the problem in their practice, and identify an evidence-based guideline. Assessing the quality and usefulness of the guideline involves examining the following: (1) the authors of the guideline; (2) the significance of the healthcare problem; (3) the strength of the research evidence; (4) the link to national standards; and (5) the cost-effectiveness of using the guideline in practice. The quality of the JNC 8 guideline is discussed using these five criteria.

FIG 13.4 Grove Model for Implementing Evidence-Based Guidelines in Practice.

Authors of the Guidelines

The panel members of the JNC 8 guideline were specifically selected from more than 400 nominees based on their “expertise in hypertension (n = 14), primary care (n = 6), … pharmacology (n = 2), clinical trials (n = 6), evidence-based medicine (n = 3), epidemiology (n = 1), informatics (n = 4), and the development and implementation of clinical guidelines in systems of care (n = 4)” (James et al., 2014, p. 508). These panel members were specifically selected based on their strong and varied expertise to develop an evidence-based guideline for management of HTN.

Significance of Healthcare Problem

James and colleagues (2014, p. 507) reported that HTN is the most “common condition seen in primary care and leads to myocardial infarction (MI), stroke, renal failure, and death if not detected early and treated appropriately. Patients want to be assured that BP treatment will reduce their disease burden, while clinicians want guidance on HTN management using the best scientific evidence.”

Strength of Research Evidence

James et al. (2014) conducted a systematic review to identify the research evidence for the national guideline for management of HTN. The participants in the studies reviewed were adults aged 18 years and older with HTN. Only the studies with strong sample sizes (> 100 participants) and adequate follow-up to yield health-related outcomes (≥ 1 year) were included in the systematic review. The panel also “limited its evidence review to only randomized controlled trials (RCTs) because they are less subject to bias than other study designs and represent the gold standard for determining efficacy and effectiveness” (James et al., 2014, p. 508).

The JNC 8 panel members had an external methodology team search the literature and summarize the data from selected studies into an evidence table (James et al., 2014). From the evidence review, the panel members developed evidence statements that provided the basis for the nine recommendations for the management of HTN. The research evidence for the development of the JNC 8 guideline for management of HTN was extremely strong.

Link to National Standards and Cost-Effectiveness of Evidence-Based Guideline

Quality evidence-based guidelines should link to national standards and be cost-effective (see Fig. 13.4). The JNC 8 evidence-based guideline for the management of HTN built on the JNC 7 national guideline for the assessment, diagnosis, and treatment of HTN. The recommendations from the JNC 7 have been supported by the Department of Health and Human Services and disseminated through the NIH (NIH, 2003, Publication No. 03-5231). Use of the JNC 8 guideline in practice is projected to be cost-effective because recommendations for the management of HTN should lead to decreased incidences of MI, stroke, CKD, HF, and cardiovascular disease (CVD)−related mortality and should improve health outcomes for adults with HTN.

Implementation of the Evidence-Based Guideline in Practice

The next step is for student nurses, RNs, and advanced practice nurses (APNs) to use the JNC 8 evidence-based guideline in their practice. Student nurses and RNs need to take and record patients’ BPs in EHRs accurately, which are part of healthcare agencies’ clinical databases (see Table 13.3). RNs and nursing students also need to educate patients and families about the lifestyle modifications needed to improve their BP, such as a regular exercise program, balanced diet, normal weight, smoking cessation, and reduced dietary salt and alcohol. Patients need to know their BP values and the BP goals desired for them. Individuals with high BP readings based on their age and chronic illnesses (e.g., diabetes, CKD) should be reported to APNs and physicians. The APNs and physicians will initiate or revise HTN medication treatments based on the health circumstances and values of their patients (see Table 13.3).

Table 13.3

Evidenced-based guideline for assessment and management of high blood pressure in adults

HIGH BP OR HTN CLASSIFICATION NURSING STUDENT AND RN INTERVENTIONS APN AND PHYSICIAN ROLE

Age (yr) HTN Diagnosis (mm Hg) Systolic BP Goal (mm Hg)a Diastolic BP Goal (mm Hg)a Lifestyle Modification Educationb DM or CKDc Pharmacological Management of HTN

< 60 ≥ 140/90 < 140 and < 90 Yes No Yes

< 60 ≥ 140/90 < 140 and < 90 Yes Yes Yes

≥ 60 ≥ 150/90 < 150 and < 90 Yes No Yes

≥ 60 ≥ 140/90 < 140 and < 90 Yes Yes Yes

Adapted from James, P. A., Oparil, S., Carter, B. L., Cushman, W. C., Denison-Himmelfard, C., Handler, J., et al. (2014). 2014 evidence-based guidelines for the management of high blood pressure in adults: Report from the panel members appointed to the Eighth Joint National Committee (JNC 8). Journal of the American Medical Association, 311(5), 507 − 520.

APN, Advanced practice nurse; BP, blood pressure; CKD, chronic kidney disease; DM, diabetes mellitus; HTN, hypertension; RN, registered nurse.

a Treatment is determined by the highest BP category, systolic or diastolic.

b Lifestyle modifications—balanced diet, regular exercise program, achieving normal weight, smoking cessation, and reduced dietary salt and alcohol; Education should be implemented for all adults with HTN.

c Patients with DM or CKD need education about management of these diseases and the link with hypertension.

The outcomes for the patient, provider, and healthcare agency need to be examined. The outcomes should be recorded in the patients’ EHRs and include the following: (1) BP readings for patients; (2) current lifestyle behaviors of patients; (3) incidence of diagnosis of HTN based on the JNC 8 guidelines; (4) appropriateness of the pharmacological therapies implemented to manage HTN; and (5) incidence of stroke, MI, HF, CKD, and CVD and related mortality over a period of 5, 10, 15, and 20 years. The healthcare agency outcomes include access to care by patients with HTN, patient satisfaction with care, and costs related to diagnosis and management of HTN, in addition to the HTN complications previously mentioned. This EBP guideline will be refined in the future based on clinical outcomes, outcome studies, and new RCTs. The use of this evidence-based guideline and additional guidelines promote EBP for nurses and healthcare agencies (see Fig. 13.4).

Introduction to evidence-based practice centers

The AHRQ launched an initiative to promote EBP by establishing 12 evidence-based practice centers (EPCs) in the United States and Canada in 1997. With this program, AHRQ became a “science partner” with public and private organizations to improve the quality, cost-effectiveness, and appropriateness of health care by synthesizing research evidence and facilitating the translation of evidence-based research findings into practice (AHRQ, 2017b).

Through the EPC program, the AHRQ awards 5-year contracts to institutions to serve as EPCs. EPCs review all relevant scientific literature on clinical, behavioral, organizational, and financial topics to produce evidence reports and technology assessments. Topics are nominated by nonfederal partners, such as professional societies, health plans, insurers, employers, and patient groups. The EPC reports are used “to inform and develop coverage decisions, quality measures, educational materials, tools, guidelines, and research agendas” (AHRQ, 2017b). The EPCs also conduct research on the methodology of systematic reviews.

The AHRQ (2017b) website provides the names of the EPCs and the focus of each center. This site also provides a link to the evidence-based reports produced by these centers. These EPCs have had an important role in developing evidence-based syntheses and guidelines for the last 2 decades.

Introduction to translational research

Some challenges to EBP have resulted in the development of a new research methodology, translational research, to improve the application of research evidence in practice. Translational research has been termed bench to bedside research, which involves the translation of basic scientific discoveries into practical applications (Callard, Rose, & Wykes, 2012). Basic research discoveries from the laboratory setting need to be tested in human studies. Also, the outcomes from human clinical trials need to be adopted and maintained in clinical practice. Translational research is being conducted in nursing and medicine to increase the implementation of evidence-based interventions in practice and determine if these interventions are effective in producing the outcomes desired.

The National Center for Advancing Translation Sciences (NCATS) was developed in 2011 as part of the NIH. To encourage researchers to conduct translational research, the NIH (2017) developed the Clinical and Translational Science Awards (CTSA) consortium. The consortium started with 12 centers located throughout the United States and expanded to 39 centers in April 2009. The program was fully implemented in 2012, with about 60 institutions involved in clinical and translational science. A website has been developed to enhance communication and encourage sharing of information related to translational research projects (see https://ncats.nih.gov/).

The CTSA Consortium is mainly focused on expanding the translation of medical research to practice. Titler (2004, p. S1) defined translational research for the nursing profession as the “Scientific investigation of methods, interventions, and variables that influence adoption of EBPs by individuals and organizations to improve clinical and operational decision making in health care. This includes testing the effect of interventions on and promoting and sustaining the adoption of EBPs.” Westra and colleagues (2015, p. 600) developed “a national action plan for sharable and comparable nursing data to support practice and translation research.” This plan provides direction for the conduct and use of translation research to change nursing practice.

Translation studies have been appearing more often in the nursing literature. For example, Mello and colleagues (2013) conducted a translation study to promote the use of an alcohol Screening, Brief Intervention, and Referral to Treatment (SBIRT) guideline in pediatric trauma centers. Before the study, only 11% of eligible patients were screened and received an intervention. “After completion of the SBIRT technical assistance activities, all seven participating trauma centers had effectively developed, adopted, and implemented SBIRT policies for injured adolescent inpatients. Furthermore, across all sites, 73% of eligible patients received SBIRT services after both the implementation and maintenance phases” (Mello et al., 2013, p. S301).

We hope that the content in this chapter has increased your understanding of EBP, the critical appraisal of research syntheses, the application of EBP models, and the implementation of EBP guidelines. We encourage you to take an active role in moving nursing toward EBP that improves outcomes for patients, nurses, and healthcare agencies.

Key Points

•  Evidence-based practice (EBP) is the conscientious integration of best research evidence with clinical expertise and patient circumstances and values in the delivery of quality, safe, cost- effective health care.

•  Best research evidence is produced by the conduct and synthesis of numerous high-quality studies in a health-related area.

•  Nurses must be knowledgeable of the benefits and challenges associated with EBP.

•  The PICO format is described for generating a clinical question to guide the use of current research evidence in practice.

•  Guidelines are provided for critically appraising the research synthesis processes of systematic review, meta-analysis, meta-synthesis, and mixed-methods systematic review.

•  A systematic review is a structured comprehensive synthesis of the quantitative research literature to determine the best research evidence available to address a healthcare question.

•  A meta-analysis is conducted to pool the results from previous studies statistically into a single quantitative analysis that provides one of the highest levels of evidence about the effectiveness of an intervention.

•  A meta-synthesis is defined as the systematic compilation and integration of qualitative studies to expand understanding and develop a unique interpretation of study findings in a selected area.

•  Reviews that include syntheses of various quantitative, qualitative, and mixed methods studies are referred to as mixed-methods systematic reviews in this text.

•  The Stetler Model of Research Utilization to Facilitate Evidence-Based Practice provides a comprehensive framework to enhance the used of research evidence by nurses in practice.

•  The Iowa Model of Evidence-Based Practice presents directions for implementing patient care based on the best research evidence and monitoring changes in practice to ensure quality care.

•  The process for developing evidence-based guidelines is described, and an example of the guideline for the assessment, diagnosis, and treatment of hypertension (HTN) is provided.

•  The Grove Model for Implementing Evidence-Based Guidelines in Practice is provided to assist nurses in determining the quality of evidence-based guidelines and the steps for using these guidelines in practice.

•  Evidence-based practice centers (EPCs), created by the AHRQ, have had an important role in the conduct of research, development of systematic reviews, and formulation of evidence-based guidelines in selected practice areas.

•  Translational research is expanding in health care to translate basic scientific discoveries into practical applications.

References