Critical Appraisal of Research

American Journal of Infection Control 48 (2020) 940−947

Contents lists available at ScienceDirect

American Journal of Infection Control

journal homepage: www.ajicjournal.org

State of the Science Review

Notice to comply: A systematic review of clinician compliance with guidelines surrounding acute hospital-based infection management

Kendall E. McKenzie MEng a,*, Maria E. Mayorga PhD b, Kristen E. Miller MSPH, DrPH c, Nishant Singh BS b, Ryan C. Arnold MD d, Santiago Romero-Brufau MD e,f

a Department of Design, North Carolina State University, Raleigh, NC b Department of Industrial and Systems Engineering, North Carolina State University, Raleigh, NC c MedStar Institute for Innovation, MedStar Health, Washington, DC d Department of Emergency Medicine, Cottage Health System, Santa Ynez, CA e Department of Medicine, Mayo Clinic, Rochester, MN f Department of Biostatistics. Harvard T.H. Chan School of Public Health, Boston, MA

Key Words:

* Address correspondence to Kendall E. McKenzie M Carolina State University, 50 Pullen Road, Raleigh, NC 276

E-mail address: [email protected] (K.E. McKenzie) Conflicts of interest: None to report. Funding: This work was supported by the N

(IIS1522072, IIS1522106, IIS1522107, IIS1833538).

https://doi.org/10.1016/j.ajic.2020.02.006 0196-6553/© 2020 Association for Professionals in Infect

Purpose: To identify and characterize studies evaluating clinician compliance with infection-related guide- lines, and to explore trends in guideline design and implementation strategies. Data sources: PubMed database, April 2017. Followed the PRISMA Statement for systematic reviews. Study selection: Scope was limited to studies reporting compliance with guidelines pertaining to the preven- tion, detection, and/or treatment of acute hospital-based infections. Initial search (1,499 titles) was reduced to 49 selected articles. Data extraction: Extracted publication and guideline characteristics, outcome measures reported, and any results related to clinician compliance. Primary summary measures were frequencies and distributions of characteristics. Interventions that led to improved compliance results were analyzed to identify trends in guideline design and implementation. Results of data synthesis: Of the 49 selected studies, 18 (37%), 13 (27%), and 10 (20%) focused on sepsis, pneumonia, and general infection, respectively. Six (12%), 17 (35%), and 26 (53%) studies assessed local, national, and international guidelines, respectively. Twenty studies (41%) reported 1-instance compliance results, 28 studies (57%) reported 2-instance compliance results (either before-and-after studies or control group studies), and 1 study (2%) described compliance qualitatively. Average absolute change in compliance for minimal, decision support, and multimodal interventions was 10%, 14%, and 25%, respectively. Twelve studies (24%) reported no patient outcome alongside compliance. Conclusions: Multimodal interventions and quality improvement initiatives seem to produce the greatest improvement in compliance, but trends in other factors were inconsistent. Additional research is required to investigate these relationships and understand the implications behind various approaches to guideline design, communication, and implementation, in addition to effectiveness of protocol impact on relevant patient outcomes. © 2020 Association for Professionals in Infection Control and Epidemiology, Inc. Published by Elsevier Inc. All

rights reserved.

Guideline adherence Practice patterns (physicians) Quality Program evaluation Professional practice gaps Outcome and process assessment (health care)

Eng, College of Design, North 07 .

ational Science Foundation

ion Control and Epidemiology, Inc. Published by Elsevier Inc. All rights reserved.

Best practice protocols are often communicated to clinicians in the form of guidelines. The Institute of Medicine formally defines clinical practice guidelines as “statements that include recommenda- tions, intended to optimize patient care, that are informed by a

systematic review of evidence and an assessment of the benefits and harms of alternative care options.”1 The primary benefit of guidelines is their potential to improve the quality of patient care.2 Common characteristics of “good” guidelines, as described in the literature, include validity, reliability, reproducibility, clinical applicability, clini- cal flexibility, clarity, multidisciplinary process, review of evidence, and documentation.3 Clinical guidelines are issued by multiple differ- ent entities (eg, public health organizations, research institutions, hospital management), have varying levels of presence across the world’s health care domain (ie, international, national, local), and are

Fig 1. Results of each phase of the review process were used to produce the final set of articles for analysis. The 2 paths depicted began with different pools of articles: the results of the original PubMed search and then the relevant references from the articles selected in the original search.

K.E. McKenzie et al. / American Journal of Infection Control 48 (2020) 940−947 941

usually communicated with an attached adherence expectation (ie, mandatory, optional).

The goal of any guideline is to provide recommendations that will translate into improve quality of patient care, and ultimately improve patient outcomes. Yet, many challenges arise when clinicians (ie, health care professionals with clinical education) attempt to put guidelines’ written, evidence-based instructions into action.4 In clini- cal scenarios, multiple competing guidelines often exist, yet little research has focused on identifying characteristics of existing guide- lines that commonly elicit compliance.5,6 Given the wide variety of clinical practice guidelines communicated and regulated by authori- tative bodies, it is not surprising that clinician’s uptake of guidelines is low.7 Success in implementation and improvement of practice seems particularly resource intensive.8 Guideline “success” is difficult to evaluate because it connects two important outcomes: clinician adherence to the guideline, and patient outcomes associated with guideline recommendations. While many studies have focused on how guideline recommendations affect patient outcomes, there has been less focus on how compliance with guidelines fits into the equa- tion. To improve compliance with guidelines, it is important to first understand how compliance with guidelines has been evaluated and achieved in previous research.

Guidelines are widely used for infectious disease care. A key example is sepsis guidelines, which have recently garnered increased international attention, due to the deadly and costly nature of the dis- ease.9-11 To inform and support current efforts toward improving such guidelines, it is valuable to investigate acute hospital-based infection guidelines in general, to understand how they have been designed, implemented, and evaluated for compliance. Although many studies report compliance with guidelines, there is a clear gap in the research toward understanding what makes a guideline suc- cessful, as well as how guidelines can be implemented to produce a higher standard of care.

We limit the scope of this systematic review to focus solely on studies that reported compliance with guidelines pertaining to the prevention, detection, and/or treatment of acute hospital-based infections. Examples of such guidelines include recommendations for antibiotic administration, pneumonia treatment, sepsis management, and hand hygiene. The objectives of this systematic review are (1) to evaluate trends in infection guideline design and implementation strategies and (2) to identify and characterize studies that have eval- uated clinician compliance and, if present, associated outcomes.

METHODS

We performed a systematic review of studies that reported clini- cian compliance with guidelines related to acute hospital-based infections, where guidelines are defined as written statements devel- oped by medical specialty societies, disease-focused organizations, or expert panels to assist practitioner and patient decisions about appropriate care under specific clinical circumstances.3 The format of this review follows the PRISMA Statement for reporting of systematic reviews that evaluate health care interventions.12 Supplemental Digi- tal Content 1 identifies locations of PRISMA components within this document.

Study eligibility

Our search was limited to studies published in the 10 years prior to our initial search date (May 13, 2016), as well as any additional articles published between the initial search date and the date collab- orators completed the article selection process (April 30, 2017). Thus, the study dates were May 13, 2006 through April 30, 2017. We looked at studies in this time period because it corresponded to the release of new international guidelines for a prominent infection:

sepsis. Studies included in the final set of articles were required to present results of clinician adherence or compliance. During the review phase, we limited our scope to only acute hospital-based infections, which were defined as acute infections (which excluded chronic or long-term infections like HIV or tuberculosis), that were also severe enough to often require a hospitalization.

Search criteria

The search was conducted in the PubMed database, using a com- bination of Medical Subject Heading (MeSH) terms, keywords, and publication types. The specific search criteria, developed with the help of a university library consultant to ensure a highly inclusive approach, returned all eligible studies (ie, those within in our publica- tion date range, written in English, and pertaining to human adults) containing information related to 4 key concepts corresponding to the population-intervention-comparison-outcome framework.13 We wanted studies that (1) focused on a population of clinicians, (2) investigated an intervention related to a guideline for acute hospital- based infections, (3) compared effects of the this intervention to the effects of not having an intervention (if a comparison was reported), and (4) evaluated their impact on the outcome of compliance. Supple- mental Digital Content 2 shows search strategy details.

Selection process

The selection process (depicted in Fig 1) involved narrowing 2 dif- ferent pools of articles into subsets (ie, Subset A and Subset B) which were then combined into our final set of articles. The first pool of articles was those returned by the original PubMed search under- went 4 phases of review (title review, abstract review, full-text review, and expert clinician review) to produce article Subset A. Rele- vant studies referenced by articles in Subset A were combined with any articles meeting our search criteria that were added to the PubMed database during the selection period for Subset A, to form a second pool of articles for review. These articles underwent the same 4 phases of article elimination, which produced article Subset B. Finally, Subsets A and B were combined to form the final set of selected studies analyzed.

A clinical expert and 2 contributor pairs participated in the study selection process. Throughout the process, contributors were instructed to include any studies that seemed likely to discuss clini- cian compliance in any way, and to err on the side of inclusion when in doubt.

Division of labor among contributor pairs was different in each review phase. In the title and abstract review phases, each article was independently reviewed by 2 contributors. Disagreements were resolved by discussion (title review) and a third tie-breaking reviewer (abstract review). During full-text review, each article was reviewed individually by a contributor. Finally, a clinical expert reviewed excluded articles from all phases to double-check other

Table 1 Characteristics of studies selected for review

Study characteristics of selected articles (N = 49) n (%)

Infection of focus Infection (general) 10 (20) Pneumonia 13 (27) Sepsis 18 (37) Other 8 (16)

Type of guideline assessed Local 6 (12) National 17 (35) International 26 (53) Surviving sepsis campaign (SSC) 17 (35)

Study region United States (US) 14 (29) Europe 18 (37) Country outside of United States/Europe 13 (27) Cross-continental 4 (8)

Publication audience Surgery 3 (6) Internal medicine 3 (6) Emergency medicine 3 (6) Critical care/ICU 14 (29) Infection/epidemiology 11 (22) Specific disease or specialty 12 (24) Health services/quality 3 (6)

Number of compliance results One-instance compliance 20 (41) Two-instance compliance 28 (57) Qualitative compliance description 1 (2)

Reporting of patient outcomes Reported patient outcomes and compliance 37 (76) Reported compliance only 12 (24)

Intervention type None 16 (33) Minimal 5 (10) Decision support 19 (39) Multimodal 9 (18)

Level of care General inpatient 15 (31) ED only 12 (24) ICU only 14 (29) Multiple levels 6 (12) Other 2 (4)

Sample size <100 3 (6) 100-9,999 28 (57) 10,000-99,999 12 (24) 100,000 or more 2 (4)

Study design Prospective only 43 (88) Retrospective and prospective 6 (12)

Study period Less than 1 year 13 (27) 1-2 years 19 (39) More than 2 years 14 (29) Not specified 3 (6)

942 K.E. McKenzie et al. / American Journal of Infection Control 48 (2020) 940−947

contributors’ work and to ensure that relevant articles were not elim- inated due to lack of clinical expertise.

Outcome measures and analysis

Details extracted from each selected study included guideline name and characteristics, compliance definition, study region, study design, publication audience, infection of focus, level of care, outcome measures reported, and any results relating to clinician compliance and/or adherence. We categorized the type of intervention employed to affect compliance. Intervention types, in order of increasing com- plexity, were none (no intervention was described or only 1 compli- ance result was presented), minimal (simple presentation of guidelines), decision support (usually electronic or paper-based), and multimodal (quality improvement (QI) initiative and/or educational program with multiple components). We also extracted specific details about the interventions used to promote guideline compli- ance, as well as results about compliance with sepsis bundles. The primary summary measures calculated in our analysis include fre- quencies and distributions of extracted information.

Risk of bias assessment

The Newcastle-Ottawa Scale for quality assessment was used to evaluate and quantify the potential for bias in compliance results reported.14 Similar to previously published studies,15 we developed a modified version of the tool (Supplemental Digital Content 3) to fit the context of this review. Each article was assessed by 2 indepen- dent reviewers and results were aggregated.

RESULTS

Study selection

The original PubMed search yielded 1,499 unique titles (Fig 1), resulting in 40 articles in Subset A. An additional 330 articles were included (see Methods), resulting in 9 articles in Subset B, for a total of 49 articles selected in the final set.

Study characteristics

Table 1 summarizes study characteristics. Of the 49 articles reviewed, 43 studies (88%) were solely prospective and 6 studies (12%) had both prospective and retrospective components. The majority of studies were located in either the United States (14 stud- ies; 29%) or Europe (18 studies; 37%). Eighteen studies (37%) focused on sepsis, 13 (27%) on pneumonia, and 10 (20%) on general infection. Most studies assessed international (26 studies; 53%) or national (17 studies; 35%) guidelines. The most common international guideline assessed was the Surviving Sepsis Campaign (17 studies; 35%). Stud- ies were disseminated to 3 primary journal audiences: critical care (14 studies; 29%), medical specialties (12 studies; 24%), and epidemi- ology (11 studies; 22%). Studies were distributed relatively evenly across general inpatient (15 studies; 31%), critical care (14 studies; 29%), and emergency (12 studies; 24%) settings. The publication dates of sepsis-related studies showed a steady increase in their prevalence throughout the duration of the eligible time window, until a sudden drop in 2015 (see Supplemental Digital Content 4). All individual study characteristics are provided in Supplemental Digital Content 5.

Compliance results

When reporting compliance results, 20 studies (41%) reported 1- instance compliance results, 28 studies (57%) reported 2-instance compliance results (either before-and-after studies or studies with a

control group), and 1 study (2%) described compliance results quali- tatively. Interventions were categorized as minimal in 5 studies (10%), decision support in 19 studies (39%), multimodal in 9 studies (18%), and none in 16 studies (33%). Definitions of compliance varied across studies, as did the methods used to measure it (eg, clinician surveys, independent record review).

The distribution of 1-instance compliance results ranged from 2% to 94% (Fig 2). When stratified by infection of focus, average compli- ance for sepsis, pneumonia, general infection, and other infections were 29%, 49%, 80%, and 61%, respectively.

Figure 3 shows the absolute change in compliance from studies that reported 2-instance compliance, organized by infection of focus (Fig 3A) and intervention type (Fig 3B). Values of initial compliance results (shown on the x-axis) ranged from 0% to 91%, values of final compliance results (shown on the y-axis) ranged from 9% to 100%,

Fig. 2. Distribution of 1-instance compliance results. The bolded result is from a study16 that only reported a compliance range, the minimum of which is displayed in the figure. Further description of the result is provided in Supplemental Digital Content 5.

Fig. 3. Absolute change in compliance percentage in studies reporting 2-instance compliance results. (A) Group results by infection of focus. (B) Group results by intervention type. Intervention types (defined in methods section) in order of increasing complexity were none, minimal, decision support, and multimodal. Note the feasible region of each graph; the maximum possible change in compliance is equal to 100 minus the initial compliance measurement. Total N = 28 + 5 + 2 = 35, since 5 studies 17-21 reported 2 two-instance com- pliance results, and 1 study22 reported 3 two-instance compliance results. Bolded markers indicate these compliance results. Definitions of these compliance results and interven- tion descriptions are provided in Supplemental Digital Content 5.

K.E. McKenzie et al. / American Journal of Infection Control 48 (2020) 940−947 943

944 K.E. McKenzie et al. / American Journal of Infection Control 48 (2020) 940−947

and the absolute change in compliance (final result minus initial result) ranged from a decrease of 6% to an increase of 51%. When grouped by infection of focus (Fig 3A), the average change in compli- ance was an improvement of 26% in sepsis-related studies, 11% in pneumonia-related studies, 15% in studies related to general infec- tion, and 16% in studies related to other infections. When grouped by intervention type (Fig 3B), the average change in compliance was an improvement of 10% for studies using minimal interventions, 14% for studies using decision support interventions, and 25% for studies using multimodal interventions.

Patient health outcomes were reported alongside compliance results in 37 studies (76%). Within these studies, the most common patient outcomes reported were mortality (36 studies; 97%) and length of stay (25 studies; 68%). Supplemental Digital Content 6 pro- vides additional patient outcome distribution details. Other outcome measures reported were clinical treatment success rate,14,15 surgical site infection incidence rate,16,17 and readmission rate.18

Bundled guidelines

Bundled guidelines appeared in more than half of the studies. Examples included a treatment bundle for ventilator-associated pneu- monia,19 centers for disease control and prevention 12-step process to reduce antimicrobial resistance,20 and most prevalently, the Surviving Sepsis Campaign bundles for sepsis resuscitation (6-hour time win- dow) and sepsis management (24-hour time window), which were evaluated for compliance in 18 studies (37%). Since 2 of these sepsis studies did not report results for an entire bundle,23,24 they were excluded from the bundle analysis. The remaining 16 articles and the bundle components for which they report compliance are shown in Table 2.

Risk of bias assessment

Conversion of the scores from the modified Newcastle-Ottawa Scale into quality levels yielded 26% good quality, 33% fair quality, and 41% poor quality, with agreement >75% between raters. The large proportion of poor-quality articles was influenced heavily by discrep- ancies surrounding the comparability of the compliance results, which was often unclear or difficult to extract due to the inclusive nature of this review. When comparability is satisfied across all articles, the adjusted quality-level distribution becomes 37% good, 43% fair, and 20% poor. Supplemental Digital Content 7 details assess- ment results.

As evidenced by the low comparability scores in the risk of bias assessment, there was substantial heterogeneity between studies, primarily associated with the level of detail used when measuring and reporting compliance. For example, studies reporting 1-instance compliance were usually not focused on this result, and so details about compliance measurement were often limited or not provided. Studies reporting 2-instance compliance varied in the length and execution of their interventions. Additionally, studies in which com- pliance was timing-based were required to define a “time-zero” when measuring compliance, whereas other studies did not include this element.

DISCUSSION

Trends in compliance achievement

From our analysis, several trends emerged that appeared to corre- late with achieving high compliance with guidelines. Trends in inter- vention type, infection of focus, and bundled guidelines are described below.

Intervention-related trends wIn general, multimodal interventions had a higher average

improvement in compliance (25%) compared to decision support inter- ventions (14%) or minimal interventions (10%; Fig 3B). Multimodal interventions were more often used in low-compliance situations as an aggressive improvement strategy. Decision support interventions (usually an electronic decision support system) seemed to be used in situations where baseline compliance was measured but could be heavily impacted by the addition of an automated system at a single complex decision point. It is also worth noting that none of the deci- sion support interventions relied on “hard-stops,” which either pre- vent the user from taking an action altogether or require the user to obtain external override of a third party in order to proceed.

Sepsis-related trends Overall, initial compliance was lower in studies of sepsis-related

guidelines (Fig. 2 and 3A), often with less than 30% compliance. How- ever, the group also showed the greatest average absolute change in compliance. This may relate to the newness of these guidelines at the time and/or the complexity of the sepsis bundles themselves. Multi- modal interventions were employed in all but one26 of the sepsis- related studies, and these interventions generally resulted in large improvements in compliance.

Bundle-related trends Compliance with bundled guidelines is difficult to track and com-

municate because it requires both standardized definitions of bundle steps across institutions and standardized definitions of time zero for temporal bundle components. The difficulty in reporting compliance with bundles is evidenced by the fact that no study in Table 2 received all checkmarks possible. Additionally, bundled compliance results can be misleading when taken at face value, since (1) bundles are tied to pay and thus more likely to be complied with and (2) compliance with the entire bundle is a binary measure and provides no information about partial compliance. In general, less-invasive bundle components correlated with higher compliance and/or better outcomes.17,27-30

Comments on decreased compliance Interestingly, 2 studies reported a decrease in compliance from

first to second instance,26,31 but neither of them were statistically sig- nificant and likely reflect random variation. Ozgun et al reported a 6% decrease in compliance with appropriate choice, dose, and duration of antimicrobial prophylaxis31; this result may have been skewed due to detailed requirements for adherence. Westphal et al reported a 4% decrease and a 2% decrease in compliance with the 6-hour and 24-hour sepsis bundles, respectively26; this was the only sepsis study to employ a minimal intervention.

Relating compliance to its impact on patient outcomes

Although the majority of studies (76%) reported patient health out- comes alongside compliance results, the primary outcome reported was mortality, a measure that can be represented in various ways32

and does not capture the full scope of the patient experience, as survival does not describe positive and negative patient health trajectories.33

There may be other useful outcome measures that would be more proximal in timing and more specific to a particular guideline. For example, studies related to sepsis guidelines could assess sepsis-specific upstream outcomes (eg, reduced organ failure, fewer ICU admissions). Twelve studies (24%) did not relate compliance back to patient health, which is the true outcome compliance aims to represent.

While tracking compliance with guidelines can be important for improving quality of care, compliance as an outcome measure has limitations that emerged in our review. First, there is no clear definition of what compliance represents, as evidenced by the

Table 2 Compliance results reported by studies that investigated Surviving Sepsis Campaign bundles

Sepsis bundle studies Compliance reported for 6-hour resuscitation bundle component* Compliance reported for 24-hour management bundle component* Compliance percentage reported for entire bundley

Lactate Blood culture

Antibiotics Fluids or vasopressors

MAP CVP SvO2 Low-dose steroids

Activated protein Cz

Glucose control

Inspiratory plateau (PiP)

-hour Bundle esult 1 Result 2)

24-hour Bundle Result 1 (Result 2)

Both bundles Result 1 (Result 2)

Almeida 2013 @ @ @ @ ‘ @ @ 2 Castellanos-Ortega 2011

@ @ @ @ @ @ @ 11

Coba 2011 @ @ @ @ @ @ @ @ @ @ @ 13 ‘ ‘ Girardis 2009 ‘ @ @ @ ‘ ‘ @ @ @ @ @ 8 (60) 8 (60) 8 (35) Jeon 2012 ‘ ‘ @ @ @ @ @ 1 (50) Kuan 2013 @ @ @ @ @ @ @ 0 (40) Levy 2010 @ @ @ @ ‘ @ @ @ @ @ @ 1 (31) 18 (36) ‘ Na 2012 @ @ @ @ @ @ @ 3 (55) Nakornchai 2014 @ @ @ @ @ @ @ 1 Nguyen 2007* @ @ @ @ @ @ @ @ @ 0 (51) Nguyen 2011 @ @ @ @ @ @ @ 0 (44) Schramm 2011 @ @ @ @ @ ‘ ‘ 3 (54) Van Zanten 2014 @ @ @ @ ‘ @ @ @ @ @ @ 0 (68) 70 (89) 38 (62) Wang 2013 @ @ @ @ ‘ @ @ @ ‘ @ @ 1 (13) 1 (9) ‘ Westphal 2011* ‘ ‘ ‘ ‘ ‘ ‘ ‘ ‘ ‘ ‘ ‘ 2 (29) 50 (48) ‘ Zambon 2008 @ @ @ @ ‘ ‘ @ @ @ @ @ 68 72 ‘

CVP, central venous pressure; MAP, Mean arterial pressure; SvO2, mixed venous oxygen saturation. @indicates that a compliance result was reported; ‘ indicates that no compliance result was reported, even though all information appeared to have been available *Bundles were broken into standard components using the summary reproduced in Levy et al.25. yAll studies reporting two-instance compliance results for entire bundles were statistically significant at the 0.05 level, except those identified by an asterisk. zThe activated protein C portion of the 24-hr management bundle was discontinued in a 2012 revision of the sepsis guidelines by Dellinger et al.9

K .E.M

cK en

zie et

al./ A m erican

Jou rn alofIn

fection C on

trol4 8 (2 0 2 0 ) 9 4 0−

9 4 7

9 4 5

6 R (

3 1

1 1

1 1 5

3

946 K.E. McKenzie et al. / American Journal of Infection Control 48 (2020) 940−947

heterogeneous definitions of compliance across studies. Second, com- pliance metrics usually imply a target goal of 100%, but this is mislead- ing, since some situations require clinicians to intentionally deviate from a guideline in order to produce the best health outcome for a patient. In fact, existing research discourages targeting 100% for any metric.4 Third, the magnitudes of improvement in compliance and patient health outcomes do not necessarily correlate. In cases where extensive and costly efforts are made to drastically improve compli- ance, there is always a chance that this effort will produce only a very small change in patient outcomes. Compliance with a protocol that does not translate to a meaningful improvement in patient outcomes is of questionable utility. Finally, all studies discussed compliance in terms of the percentage of patient cases that were compliant across all providers and did not compare compliance distributions between pro- vider types or individual providers. Further analysis related to pro- viders may be informative in guideline design and implementation.

Sustaining compliance

Following any intervention related to compliance with infection guidelines, there should be considerations for sustainability. How- ever, in the risk of bias assessment, contributors agreed that adequate consideration was given for sustainability (meaning measured at least 6 months after the intervention) in only 35 studies (71%). One of the biggest factors in sustainability is cost of an intervention, but only 5 studies discussed this factor.16,20,31,34,35 Additionally, key opera- tional considerations (eg, training time required, effect on workload) were ignored almost completely. One study surveyed physicians to investigate specific barriers (eg, not sure how to do it, patient refused, forgot to do it) to guideline compliance.29

Bias and limitations

The authors note there is a risk of bias both within and across the studies included in this review. Some common themes emerged in the limitations of compliance-related studies, such as varying defini- tions of “time zero” for temporal interventions. The potential for reporting bias is also evident, since a negative effect on compliance was reported in only 2 studies26,31; other studies may have also pro- duced negative results but did not report them.

The primary limitation of our review relates to the initial search strategy. Under the guidance of a university librarian subject-matter consultant, we primarily used MeSH terms (instead of keywords) in our search structure because they are assigned systematically by the PubMed database and represent the main ideas of each study. Thus, any studies with peripheral connection to our main ideas were not returned in our initial search. We note that another review article iden- tified additional PubMed studies reporting sepsis bundle compliance results, due primarily to the use of keywords in their search strategy.36

Although our respective analyses agree (for the articles we both selected), this brings up concerns about relying on database-specific categorization as a true representation of study content. One final limi- tation of this study was the English-only language restriction.

CONCLUSIONS

In this systematic review of studies that reported compliance for the prevention, detection, and/or treatment of acute hospital-based infections. Multimodal interventions (often including quality improvement initiatives) seem to result in the greatest improvement in compliance (25% improvement), followed by decision support (14%). More research is required to understand the implications of different approaches to guideline design, communication, and imple- mentation, especially in relation to patient outcomes and various cli- nician types.

Acknowledgments

This work was part of a collaboration among North Carolina State University, Mayo Clinic, Drexel University, and MedStar Health. The authors wish to thank Joshua Barclay for his contributions to the arti- cle selection process.

SUPPLEMENTARY MATERIALS

Supplementary material associated with this article can be found in the online version at https://doi.org/10.1016/j.ajic.2020.02.006.

References

1. Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. In: Graham R, Mancher M, Miller Wolman D, eds. Clin- ical Practice Guidelines We Can Trust. Washington (DC): National Academies Press (US); 2011.

2. Woolf SH, Grol R, Hutchinson A, et al. Potential benefits, limitations, and harms of clinical guidelines. BMJ. 1999;318:527–530.

3. Institute of Medicine (US) Committee to Advise the Public Health Service on Clini- cal Practice Guidelines. In: Field MJ, Lohr KN, eds. Clinical Practice Guidelines: Direc- tions for a New Program. Washington (DC): National Academies Press (US); 1990.

4. Young RA, Roberts RG, Holden RJ. The challenges of measuring, improving, and reporting quality in primary care. Ann Fam Med. 2017;2:175–182.

5. Greenfield S, Kaplan SH. When clinical practice guidelines collide: finding a way forward. Ann Intern Med. 2017;167:677–678.

6. Shekelle PG. Clinical practice guidelines: what's next? JAMA. 2018;320:757–758. 7. Upshur RE. Do clinical guidelines still make sense? No.. Ann Fam Med. 2014;12:202–

203. Erratum in: Ann Fam Med 2014; 12:301. 8. Davenport TH. Foreword. In: McNeill D, ed. Analytics in Healthcare and the Life Sci-

ences: Strategies, Implementation Methods, and Best Practices. New Jersey: Pearson; 2014:xiii–xvii.

9. Dellinger RP, Levy MM, Rhodes A, et al. Surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med. 2013;41:580–637.

10. Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sep- sis-3). JAMA. 2016;315:762–774.

11. Singer M, Deutschman CS, Seymour CW, et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315:801–810.

12. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting system- atic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6: e1000100.

13. Schardt C, Adams MB, Owens T, et al. Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med Inform Decis Mak. 2007;7:16.

14. Wells GA, Shea B, O’Connell D, et al: The Newcastle-Ottawa scale (NOS) for assess- ing the quality of nonrandomised studies in meta-analyses. Available at: http:// www.ohri.ca. Accessed January 26, 2019.

15. Rotenstein LS, Torre M, Ramos MA, et al. Prevalence of burnout among physicians: a systematic review. JAMA. 2018;320:1131–1150.

16. Barlow G, Nathwani D, Williams F, et al. Reducing door-to-antibiotic time in com- munity-acquired pneumonia: controlled before-and-after evaluation and cost- effectiveness analysis. Thorax. 2007;62:67–74.

17. Schramm GE, Kashyap R, Mullon JJ, et al. Septic shock: a multidisciplinary response team and weekly feedback to clinicians improve the process of care and mortality. Crit Care Med. 2011;39:252–258.

18. Dambrava PG, Torres A, Vall�es X, et al. Adherence to guidelines' empirical antibi- otic recommendations and community-acquired pneumonia outcome. Eur Respir J. 2008;32:892–901.

19. Schok T, Simons PC, Janssen-Heijnen ML, et al. Prospective evaluation of the added value of imaging within the Dutch National Diagnostic Appendicitis Guideline—do we forget our clinical eye? Dig Surg. 2014;31:436–443.

20. de Almeida SM, Marra AR, Wey SB, et al. Implementation of an antibiotic prophy- laxis protocol in an intensive care unit. Am J Infect Control. 2012;40:721–725.

21. Schwann NM, Bretz KA, Eid S, et al. Point-of-care electronic prompts: an effective means of increasing compliance, demonstrating quality, and improving outcome. Anesth Analg. 2011;113:869–876.

22. Conner BT, Kelechi TJ, Nemeth LS, et al. Exploring factors associated with nurses' adoption of an evidence-based practice to reduce duration of catheterization. J Nurs Care Qual. 2013;28:319–326.

23. Bloos F, Thomas-R€uddel D, R€uddel H, et al. Impact of compliance with infection management guidelines on outcome in patients with severe sepsis: a prospective observational multi-center study. Crit Care. 2014;18:R42.

24. Men�endez R, Torres A, Reyes S, et al. Initial management of pneumonia and sepsis: factors associated with improved outcome. Eur Respir J. 2012;39:156–162.

25. Levy MM, Dellinger RP, Townsend SR, et al. The Surviving Sepsis Campaign: results of an international guideline-based performance improvement program targeting severe sepsis. Crit Care Med. 2010;38:367–374.

K.E. McKenzie et al. / American Journal of Infection Control 48 (2020) 940−947 947

26. Westphal GA, Koenig �A, Caldeira Filho M, et al. Reduced mortality after the imple- mentation of a protocol for the early detection of severe sepsis. J Crit Care. 2011;26:76–81.

27. Asadi L, Eurich DT, Gamble JM, et al. Impact of guideline-concordant antibiotics and macrolide/b-lactam combinations in 3203 patients hospitalized with pneumo- nia: prospective cohort study. Clin Microbiol Infect. 2013;19:257–264.

28. Almeida M, Ribeiro O, Arag~ao I, et al. Differences in compliance with Surviving Sepsis Campaign recommendations according to hospital entrance time: day ver- sus night. Crit Care. 2013;17:R79.

29. Wang Z, Xiong Y, Schorr C, Dellinger RP. Impact of sepsis bundle strategy on out- comes of patients suffering from severe sepsis and septic shock in china. J Emerg Med. 2013;44:735–741.

30. Zambon M, Ceola M, Almeida-de-Castro R, et al. Implementation of the Surviving Sepsis Campaign guidelines for severe sepsis and septic shock: we could go faster. J Crit Care. 2008;23:455–460.

31. Ozgun H, Ertugrul BM, Soyder A, et al. Peri-operative antibiotic prophylaxis: adher- ence to guidelines and effects of educational intervention. Int J Surg. 2010;8:159–163.

32. Walkey AJ, Lindenauer PK. Keeping it simple in sepsis measures. J Hosp Med. 2017;12:1019–1020.

33. Venkatesh B, Finfer S, Cohen J, et al. Adjunctive glucocorticoid therapy in patients with septic shock. N Engl J Med. 2018;378:797–808.

34. Arboe B, Laub RR, Kronborg G, Knudsen JD. Evaluation of the decision support sys- tem for antimicrobial treatment, TREAT, in an acute medical ward of a university hospital. Int J Infect Dis. 2014;29:156–161.

35. Rimawi RH, Mazer MA, Siraj DS, et al. Impact of regular collaboration between infectious diseases and critical care practitioners on antimicrobial utilization and patient outcome. Crit Care Med. 2013;41:2099–2107.

36. Damiani E, Donati A, Serafini G, et al. Effect of performance improvement pro- grams on compliance with sepsis bundles and mortality: a systematic review and meta-analysis of observational studies. PLoS One. 2015;10: e0125827.