Evaluation of Outcomes

Erickson_et_al-2015-TranslationofobesitypracticeguidelinesMeasurementandevaluation..pdf

Home >Nursing homework help >Evaluation of Outcomes

POPULATIONS AT RISK ACROSS THE LIFESPAN: PROGRAM EVALUATIONS

Translation of Obesity Practice Guidelines: Measurement and Evaluation Kristin J. Erickson, MS, RN, PHN,1 Karen A. Monsen, PhD, RN, FAAN,2 Ingrid S. Attleson, BSN, RN,3

David M. Radosevich, PhD, RN,4 Gary Oftedahl, MD,5 Claire Neely, MD,6 and Diane R. Thorson, MS, RN, PHN7 1Health Initiatives and Evaluation, Otter Tail County Public Health and PartnerSHIP 4 Health, Fergus Falls, Minnesota; 2School of Nursing Affiliate Faculty, Institute for Health Informatics, University of Minnesota School of Nursing, Minneapolis, Minnesota; 3School of Public Health, University of Minnesota, Minneapolis, Minnesota; 4Clinical Outcomes Research Center, University of Minnesota, Minneapolis, Minnesota; 5Institute for Clinical Systems Improvement, Bloomington, Minnesota; 6Institute for Clinical Systems Improvement, Bloomington, Minnesota; and 7Public Health Department, Otter Tail County, Fergus Falls, Minnesota

Correspondence to:

Kristin Erickson, Otter Tail County Public Health, PartnerSHIP 4 Health Project Evaluator, 560 Fir Ave West, Fergus Falls, MN 56537.

E-mail: [email protected]

ABSTRACT Objective(s): A public health nurse (PHN) in the Midwestern United States (U.S.) led a collaborative system-level intervention to translate the Institute of Clinical Systems Improvement (ICSI) Adult Obesity Guideline into interprofessional practice. This study (1) evalu- ated the extent of guideline translation across organizations and (2) assessed the Omaha System as a method for translating system-level interventions and measuring outcomes. Design and Sam- ple: This retrospective, mixed methods study was conducted with a purposeful sample of one administrator (n = 10) and two to three clinicians (n = 29) from each organization (n = 10). Measures: Omaha System Problem Rating Scale for Outcomes Knowledge, Behavior, and Status (KBS). KBS ratings gathered from semi-structured interviews and Omaha System documentation were analyzed using standard descriptive and inferential statistics and triangulated findings with participant quotes. Results: KBS ratings and participant quotes revealed intervention effectiveness in creating sustained system-level changes. Self-reported and observed KBS ratings demonstrated improvement across organizations. There was moderate to substantial agreement regarding bench- mark attainment within organizations. On average, self-reported improvement exceeded observer improvement. Conclusions: System-level PHN practice facilitator interventions successfully translated clinical obesity guidelines into interprofessional use in health care organizations. The Omaha System Problem Rating Scale for Outcomes reliably measured system-level outcomes.

Key words: adult, health care reform, health promotion, obesity, Omaha system, practice guidelines as topic, primary health care, program evaluation, public health practice, public-pri- vate sector partnerships, standardized terminology.

Background Obesity prevalence continues to escalate in the Uni- ted States (U.S.) with associated increases in health care costs and social burden (Ogden, Carroll, Kit, & Flegal, 2012). The Centers for Disease Control and

Prevention (CDC) estimates that 35.1% of adult Americans are obese and an additional 33.9% are overweight (CDC, 2014). Overweight and obese individuals risk suffering from coronary heart disease, Type 2 diabetes, cancer, hypertension,

222

Public Health Nursing Vol. 32 No. 3, pp. 222–231

dyslipidemia, stroke, liver and gallbladder disease, sleep apnea and respiratory problems, osteoarthri- tis, and gynecological problems (National Institute of Health [NIH], 2000). Researchers estimate obes- ity-related health care costs at $147 billion in 2008 (Finkelstein, Trogdon, Cohen, & Dietz, 2009).

U.S. health care organizations and clinicians are challenged to fully implement obesity guide- lines (Felix, West, & Bursac, 2008; Strategies to Overcome and Prevent Obesity (STOP) Alliance, 2010). Numerous robust clinical guidelines are available. The National Heart, Lung, and Blood Institute (NHLBI) in cooperation with the NIH published the first federal clinical obesity guide- lines in 1998. This guideline recommended a two- step process: determination of risk status and motivation to lose weight, followed by treatment management (NHLBI, 1998; NIH, 2000; Schuster, Tasosa, & Terwoord, 2008). In 2013, the American College of Cardiology, American Heart Association, and The Obesity Society released an update of this guideline (Jensen et al., 2013). The United States Preventive Services Task Force (USPSTF) released clinical guidelines in 2003. The updated 2012 US- PSTF guideline advocates screening all adult patients for obesity and offering intensive, multi- component behavioral interventions to all obese patients (USPSTF, 2012). The first edition of the Institute for Clinical Systems Improvement (ICSI) Prevention and Management of Obesity for Adults Guideline was published in 2004 with its sixth edi- tion released in 2013 (Figure 1). The ICSI guideline recommends addressing obesity with patients using a combination of two approaches, the 5As frame- work and motivational interviewing. The 5As framework consists of five components: Ask, Advise, Assess, Assist, and Arrange. Ask refers to screening (annually and as needed). Advise is a patient-centered conversation regarding risks of obesity and benefits of a healthy lifestyle. Assess refers to determining readiness for change. Assist refers to helping the patient set goals. Arrange refers to scheduling follow-up appointments (Fitch et al., 2013). Research shows that this framework facilitates individual as well as system-level behav- ior change (Glasgow, Goldstein, Ockene, & Pronk, 2004). Motivational interviewing, a behavioral counseling method originally developed for the addictions field and shown effective for weight loss, is adaptable to the primary care setting

(Armstrong et al., 2011; Fitch et al., 2013; Hall, Gibbie, & Lubman, 2012).

The successful translation of any evidence- based clinical guideline requires system-level adap- tations in addition to changes by individual health care professionals (Ploeg, Davies, Edwards, Gifford, & Miller, 2007). According to Keller, Strohschein, Lia-Hoagberg, and Schaffer, “Population-based sys- tem-focused practice changes organizations, poli- cies, laws, and power structures. The focus is on the systems that impact health, not directly on indi- viduals and communities” (Keller, Strohschein, Lia- Hoagberg, & Schaffer, 2004, p. 457). Examples of system-level adaptations that support the imple- mentation of clinical obesity guidelines include:

• Designation of a person or department responsi- ble for timely implementation and monitoring (Poobalan, Aucott, Ahmed, & Smith, 2010)

• Incorporation of quality improvement systems such as chart prescreening, risk assessment forms, prompts, flow-sheets, reminder/recall sys- tems, patient education materials, and/or redis- tributing responsibilities among office staff (Bordley, Margolis, Stuart, Lannon, & Keyes, 2001; Krist et al, 2008; Schriefer, Landis, Tur- bow, & Patch, 2009).

• Selection of a uniform protocol such as the 5As for describing, delivering, and evaluating health behavioral counseling interventions (Goldstein, Whitlock, & DePue, 2004; Jay et al, 2010).

• Integration of staff into practice teams (Pimlott, 2008) In larger settings, teams consist of multi- ple disciplines including physicians, nurse practi- tioners, physician assistants, nurses, medical assistants, health educators, behavioral health counselors, pharmacists, and/or other office staff, while in smaller settings teams consist of a clinician and perhaps one other member (Bodenheimer, 2007).

• Development of counseling skills via interactive, step-based continuing education learning oppor- tunities during workshops, small groups, and/or individual training sessions (Davis et al, 1999).

• Utilization of an implementation process tailored to the practice (Baskerville, Liddy, & Hogg, 2012; Ploeg et al, 2007).

• Adoption of a comprehensive worksite wellness policy that supports clinician health Normal

Erickson et al.: Translation of Obesity Practice Guidelines: Measurement 223

Figure 1. Institute for Clinical Systems Improvement (ICSI) Prevention and Management of Obesity for Adults Guideline Algorithm

224 Public Health Nursing Volume 32 Number 3 May/June 2015

weight physicians and nurses are more apt than their overweight counterparts to address obesity with their patients (Zhu, Norman, & While, 2011).

Superior outcomes result when multiple individual- and system-level changes occur simultaneously (Derksen et al., 2012; Gowin, Avonts, Horst-Si- korska, Dytfeld, & Michalak, 2012).

Health risks related to obesity make decreasing the high prevalence of obesity a public health prior- ity (Health & Human Services, 2010). Both the Institute of Medicine (IOM) (2012) and the Afford- able Care Act (Community Benefit, 2014) encourage cooperative and collaborative partnerships between public health, primary care, and other community partners. Neither one, however, addresses the role or potential role that public health plays in the arena of collaborative clinical obesity guideline implementation.

Research regarding guideline implementation supports the role of the practice facilitator, finding that nurses are effective practice champions for preventive health services within their respective health care setting (Baskerville et al., 2012; Holtrop et al., 2009). In a system-level obesity prevention and management intervention, a public health nurse (PHN) acted as practice facilitator of a sys- tem-level intervention, translating clinical obesity guidelines within public health, primary care, and other health care settings. This article reports mea- surement and outcomes evaluation of the study. Clinician perspectives regarding reasons for transla- tion success are reported elsewhere (Monsen et al., 2014). The purposes of this study were (1) to evalu- ate the extent of guideline translation across orga- nizations and (2) to assess the Omaha System as a method for translating system-level interventions and measuring outcomes.

Methods

Design and sample A university-community research team conducted this retrospective, mixed methods research. This study was approved by the institutional review boards and leadership of the University of Minne- sota and local public health departments in Becker, Clay, Otter Tail, and Wilkin counties in Minnesota. Study participants consented to interviews.

Four rural Midwestern U.S. county-level public health departments conducted a local assessment and discovered rising aggregate obesity rates from 25.2% in 2004 to 29.4% in 2010, exceeding the 2010 state average of 24.8% (CDC, 2013). These four public health departments received state fund- ing to address obesity through policy, system, and environmental interventions in health care, work- sites, communities, and schools. From a menu of options, they chose to promote the translation of the ICSI Prevention and Management of Obesity for Adults Guideline into clinical practice (Fitch et al., 2013). To achieve that goal, they assigned the role of intervention practice facilitator to a PHN from one of the agencies.

In 2010, the PHN practice facilitator recruited partners from health care organizations in the four county regions by fax, e-mail, telephone, and face- to-face conversations with administrators and qual- ity improvement personnel. In total, she recruited five primary care clinic partners and five additional health care partners (four county-level public health agencies and one independent physical and occupa- tional therapy (PT/OT) clinic). Partners represented two major health care systems, a migrant health service, and a federally qualified health care center. Most partners served rural areas (70%) and received public funding (60%). Each partner orga- nization selected a physician/director champion to lead an interprofessional team consisting of repre- sentation from at least 2–3 of the following areas: nursing, dietetics, physical therapy, quality improvement, information technology, and coding and billing.

In the first 12 months, ICSI consultants and PHN practice facilitator led partners in a learning collaborative utilizing face-to-face and web-based interactive trainings. Partners learned about the obesity guideline, organizational readiness to change, quality improvement strategies, adaptive leadership, patient-centered and patient-empower- ing conversational style and spirit (motivational interviewing), as well as how to develop an action plan with measurable aims. The consultants and PHN practice facilitator-elicited end-user input to guide the creation of site-specific action plans based on a generic menu of aims and measures encouraging policy, system, and environmental changes. The generic menu and sample action plan are available online (Thorson, Erickson, Attleson, &

Erickson et al.: Translation of Obesity Practice Guidelines: Measurement 225

Monsen, 2014). Partner organizations incorporated system-level interventions such as the following into quarterly action plans:

• Designation of a champion and a multidisciplin- ary team responsible for guideline implementa- tion,

• Facilitation of interprofessional training regard- ing the guideline, adaptive challenges, and moti- vational interviewing,

• Relocation of scales to private locations and placement of working stadiometers conducive to work flow to facilitate body mass index (BMI) screening,

• Development of a team approach to efficiently and effectively deliver the 5As,

• Addition of prompts and reminders to the medi- cal record system,

• Integration of obesity-related teaching tools and resources (body mass index brochures, patient action plan template, food/activity logs, portion control plates/handouts, home exercise routines, calorie counters, clinic/community resource bro- chures, and food and fat models) into the patient visit,

• Development of electronic tracking systems for panel or population management,

• Adoption and implementation of employee well- ness initiatives or worksite wellness policies

The consultants held conference calls and webinars to provide follow-up and support, and to conduct evaluation.

Following the initial 12-month intervention, the PHN practice facilitator met quarterly with each partner organization to assess action plan progress and barriers to success, and to offer guidance and resources supportive of guideline implementation. The PHN practice facilitator applied the 5As frame- work to her system-level facilitation activities: Ask refers to screening partner agency for obesity/over- weight protocol and worksite wellness initiative/ policy. Advise refers to advising partner agency of screening results and implications. Assess refers to assessment of partner agency’s readiness to change. Assist refers to helping partner agency with goal- setting, action plan creation, and connecting to resources. Arrange refers to scheduling follow-up meetings to review action plan progress, address barriers and update action plan. The expanded version of the 5As framework used by the PHN

Practice Facilitator, as well as the intervention timeline, is available online (Thorson et al., 2014).

Study participants (n = 39) comprised a conve- nience sample from the 10 partner sites consisting of an administrator from each site (n = 10), three clinicians from nine of the 10 sites and two clini- cians from one site (n = 29). Clinicians included 12 PHNs, five registered nurses, four nurse practitio- ners, two physicians, two physician assistants, and one each of the following: registered dietitian, phys- ical therapist, occupational therapist, and physical therapy assistant. A research assistant conducted interviews over a 6-month period between October 2012 and April 2013.

Measures The research team used the Omaha System to docu- ment systems-level practice and measure outcomes (Martin, 2005). The Omaha System Intervention Scheme describes health care interventions using a hierarchy of defined actions and targets. The PHN practice facilitator developed an evidence-based care plan for practice facilitation based on the ICSI guideline using the Intervention Scheme. Based on the Omaha System sign/symptom “inadequate treatment plan,” she chose the Omaha System Prob- lem Health care supervision, defined as the “man- agement of health care treatment by health care providers” (Martin, 2005, pp. 346–347). This care plan is available online (Thorson et al., 2014).

The Omaha System Problem Rating Scale for Outcomes consists of three 7-point Likert-type ordi- nal scales for measuring the range of severity for the dimensions of Knowledge, Behavior, and Status (KBS) relative to each problem. In measuring sys- tem-level outcomes, the challenge is to define whose knowledge, behavior, and status is measured. For this study, we interpreted organizational change through measuring organizational knowledge, behavior, and status as reported by an outside observer (PHN facilitator) as well as participants who were administrators or clinicians within the organizations. In this intervention, Knowledge refers to “knowledge related to the content of the clinical obesity guideline,” and outcomes range from “no knowledge” to “superior knowledge.” Behavior refers to the “implementation of the clinical obesity guideline,” and outcomes range from “never imple- menting” to “consistent implementation.” Status refers to the “adoption of the clinical obesity

226 Public Health Nursing Volume 32 Number 3 May/June 2015

guideline,” and outcomes reflect a continuum from “no adoption (extreme signs/symptoms)” to “gener- alized adoption (no signs/symptoms).” The KBS rat- ing guidance developed for this intervention is available online (Thorson et al., 2014). The PHN practice facilitator assigned KBS ratings to each partner organization preintervention (baseline) and at the 3-year point of the intervention (follow-up). The researchers selected a rating of three as the out- come benchmark for the three outcome dimensions of Health Care Supervision.

A single independent interviewer conducted semi-structured interviews with participants, begin- ning at the 3-year point of the intervention. To obtain partner organization self-KBS ratings from participants, the interviewer asked clinicians to rate themselves, and the administrators to rate the orga- nization, regarding the translation of the ICSI adult clinical obesity guideline. The researchers abstracted KBS ratings from the interview tran- scripts. Comparison of numeric results with the interview narratives to aggregated KBS ratings enabled assessment of the validity of the Omaha System Problem Rating Scale for Outcomes results.

Analytic strategy Descriptive analysis of KBS ratings included com- parisons of baseline, follow-up, and change scores. The percentage of agreement across raters for each organization evaluated benchmark attainment. In- traclass Correlation (ICC) analysis compared the judgments of the raters (PHN practice facilitator, administrators, and clinicians) respective to each setting (Landis & Koch, 1977). A nested analysis pair-by-pair comparison of the PHN practice facili- tator rating with administrator and clinician ratings adjusted for random error (Koch, 1982).

Results

This study aimed to evaluate the extent of guideline translation across organizations. Outcome Analysis revealed that on a scale of 1–5 (1 = low/negative; 5 = high/positive), the average KBS ratings across partner organizations increased over two points from baseline to 3 years follow-up. Knowledge rat- ings increased from 1.20 (no knowledge) to 3.32 (basic knowledge). Behavior ratings increased from 1.10 (not appropriate) to 3.56 (between inconsis- tently—consistently appropriate). Status ratings

increased from 1.10 (extreme signs/symptoms) to 3.85 (minimal signs/symptoms) (Table 1).

Nine of 10 organizations attained average KBS benchmarks of 3 and seven organizations attained a Status benchmark of 4. The percentage of agree- ment regarding benchmark attainment (rating ≥3) was 83.9% (K), 91.1% (B), and 89.4% (S). Knowl- edge benchmark attainment showed substantial agreement, while Behavior and Status benchmark attainment showed moderate agreement (Table 2).

Comparison of PHN practice facilitator observed follow-up ratings and partner self-reported follow-up ratings revealed that partner ratings were slightly lower on average for Knowledge (3.90 vs. 3.18) and higher on average for Behavior (3.00 vs. 3.70) and Status (2.80 vs. 4.13) (Table 1 and Fig- ure 2). The ICC analysis demonstrated moderately strong agreement for Knowledge and weaker agree- ment for Behavior and Status (Table 3).

Responses to semi-structured interview ques- tions aided in interpreting these findings: the par- ticipant comments matched general trends in Omaha System documented practice ratings. Refer- ences to information and education revealed an increase in guideline-related knowledge:

We’ve had three of us that have gone to ICSI’s Collaborative meetings that we have done, and we have also participated in on-site and webinar information through ICSI, and through [the col- laborative quarterly meetings with the PHN] . . . I think each of those have given information to help us get to where we’re at right now.

You know, the [collaborative was] very supportive —they’ve given us a lot of education, they’ve given us a lot of resources, just kind of organized that information for us, supported our staff, sharing best practices among other clinics with us, which we have found to be very valuable.

Other statements illustrated an increase in guideline-related behavior:

I think, when I first started out with [the collabo- rative], I thought, “This was such a huge task, how do you even begin to chip away at it?” But, as I look back, we are making differences . . . all these little changes, I think, are really making a big difference. And because [the collaborative] got us going, I think it’s making a difference. I think we have a long ways to go.

. . .because we’ve had [the PHN] coming, checking in with us, and reminding us of things, and

Erickson et al.: Translation of Obesity Practice Guidelines: Measurement 227

reviewing previously set goals, and assisting us in setting future goals . . . it’s helped us stay on track, and it’s helped us continue to be mindful of the process.

Additional comments highlighted an increase in guideline-related status:

We have embedded this in all of our client records, so that that becomes part of our conver- sations over either one visit or multiple visits dur- ing the time that we know them. It’s our policy that those be assessed and recorded.

So before . . . it would not necessarily be addressed, but because of the fact that it’s now our agency policy that we address nutrition, physi- cal activity, and substance use on every single cli- ent that we see . . . OK, so I mean . . . it’s right up front, and it’s in your face, and you do that.

Participants also described reasons for success- ful guideline translation, reported separately (Mon- sen et al., 2014).

This study also aimed to assess the Omaha Sys- tem as a method for translating system-level inter- ventions and measuring outcomes. The guideline translation care plan, KBS rating guide, and study findings demonstrate the utility of the Omaha Sys- tem at the system-level for intervention description and outcome measurement.

Discussion

A PHN-facilitated interprofessional system-level translation of the ICSI adult obesity guideline into clinical practice resulted in increased Knowledge,

Behavior, and Status ratings, demonstrating effec- tiveness of the intervention. The magnitude of the increase was consistent with a large effect size (Johnson, McMorris, Raynor, & Monsen, 2013). This innovative PHN-facilitated collaboration between public health and partner organizations across the multifaceted health care system demon- strates successful partnership between public health, primary care, and other health care organi- zations regarding obesity management.

The collaborative effort intentionally broadened the definition of a clinical guideline implementation setting beyond a primary care setting. This strategy enabled implementation of the ICSI guideline in public health departments and an independent physical/occupational therapy clinic as well as in traditional primary care settings. Furthermore, it increased the guideline scope, making possible a more comprehensive and systematic population- level approach to obesity across multiple health care delivery settings.

The Omaha System Care Plan successfully described the system-level intervention. The PHN considered each health care organization as the cli- ent and used the care plan to document system- level work that focused on system-level changes. This novel use of the Omaha System guided sys- tem-level practice, captured system-level interven- tions, and measured system-level outcomes, effectively managing system-level practice and out- comes. Future interventions using the Omaha Sys- tem at the system-level should integrate policy, system, and environmental elements into the

TABLE 1. Average, Range, and Standard Deviation Related to Omaha System Outcome Ratings by Public Health Nurse Practice Facilitator (n = 1), Administrator (n = 10), Clinician (n = 29), and Partner Organization (n = 10) at Baseline and Follow-up (33–39 months from baseline)

Baseline or follow-up rating Rater

Omaha System outcome ratings

Knowledge Behavior Status

Average Range SD Average Range SD Average Range SD

Baseline PHN Facilitator 1.20 1.00–3.00 0.63 1.10 1.00–2.00 0.32 1.10 1.00–2.00 0.32 Follow-up PHN Facilitator 3.90 3.00–5.00 0.74 3.00 2.00–4.00 0.82 2.80 2.00–4.00 0.63

Administrator 3.30 2.00–4.00 0.67 3.40 2.00–4.00 0.84 3.57 1.00–5.00 1.30 Clinician 3.15 2.33–5.00 0.76 3.80 3.33–4.00 0.28 4.27 3.00–5.00 0.60 Organization 3.18 2.50–4.75 0.62 3.70 3.00–4.00 0.35 4.13 2.50–4.67 0.65 All 3.32 2.60–4.60 0.55 3.56 2.80–4.00 0.37 3.85 2.50–4.67 0.60

Note. SD = standard deviation; PHN = public health nurse.

228 Public Health Nursing Volume 32 Number 3 May/June 2015

Omaha System Care Plan in a manner that facili- tates reporting requirements related to funding reports or administrative requirements. Further research should examine patient-level outcomes in relationship with system-level outcomes as well as evaluate other ways to measure and validate sys- tem-level outcomes, such as chart audits or popula- tion health indicators that show improvement relative to the observed organizational changes.

This study had limitations common to observa- tional studies and convenience samples. Adminis- trator, facilitator, and clinician ratings of Knowledge, Behavior, and Status reflect each per- son’s perspective and inherent bias. Without ran- domization or use of a control group, researchers cannot assign causation for changes in guideline knowledge and use. However, participants reported reasons for successful guideline implementation (Monsen et al., 2014). Furthermore, the lengthy intervention increased the possibility of other fac- tors influencing intervention success. Future studies should address these methodological issues. This is the first study that has compared observer ratings to self-ratings. Subsequent research should develop methods for validating observational data in clinical settings.

The National Public Health Performance Stan- dards Program directs public health to ensure a competent workforce through capacity-building opportunities (CDC, 2008–2013; Internal Revenue Service, 2013). This study demonstrates the devel- opment of public health nurses as practice facilita- tors in health care settings other than public health, and suggests that there is potential for pub-

lic health to implement clinical obesity guidelines within public health programs such as Women, Infants and Children (WIC), the Nurse-Family Part- nership (NFP), and Minnesota Senior Health Options (MSHO). System-level PHN practice facilitator interventions successfully translated clinical obesity guidelines into interprofessional use in health care organiza- tions. The Omaha System KBS outcome ratings provided reliable data to measure system-level out- comes. This research contributes to best practice recommendations related to the interprofessional translation of clinical adult obesity guidelines into reality across diverse and numerous health care set- tings. In addition, it demonstrates potential for use of the Omaha System as a means of guiding and evaluating system-level interventions, and paves the way for intentional collaborative partnerships between public health, primary care, and other community health care partners.

Acknowledgments

This project was supported by Grant Number 1UL1RR033183-01 from the National Center for Research Resources (NCRR) and by Grant Number 8UL1TR000114-02 from the National Center for Advancing Translational Sciences (NCATS) of the National Insti- tutes of Health (NIH) to the University of Minnesota Clinical and Translational Science Institute (CTSI); Otter Tail County Public Health (OTCPH); the Minnesota Department of Health Statewide Health Improvement Program (SHIP) and PartnerSHIP 4 Health. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the Minnesota Department of Health, the CTSI or the NIH. The University of Minnesota CTSI is part of a national Clinical and Translational Science Award (CTSA) consortium created to accelerate laboratory discoveries into treatments for patients. The authors acknowledge the scientific methods support of the Omaha System Partnership for Knowledge

TABLE 2. Percent Agreement Met (Rating of 3–5) and Unmet (Rating of 1–2) between Public Health Nurse Practice Facil- itator (n = 1), Administrator (n = 10), and Clinician (n = 29) for Omaha System Follow-up Outcome Ratings and Cohen’s Kappa Adjusted for Random Agreement and Corresponding Landis and Koch Guideline

Outcome rating concepts

Percent agreement in follow-up ratings between PHN practice facilitator, administrators, and clinicians

Adjusted Cohen’s Kappa

Landis and Koch guidelineAdministrators

First clinician from

each agency

Second clinician from

each agency

Third clinician from

each agency Total

Knowledge 90 80 60 89 82 0.64 Substantial agreement

Behavior 70 78 67 67 70 0.41 Moderate agreement

Status 80 70 70 78 77 0.54 Moderate agreement

Note. PHN = public health nurse.

Erickson et al.: Translation of Obesity Practice Guidelines: Measurement 229

Discovery and Health Care Quality, Center for Nursing Informatics, University of Minnesota.

References

Armstrong, M. J., Mottershead, T. A., Ronksley, P. E., Sigal, R. J., Campbell, T. S., & Hemmelgarn, B. R. (2011). Motivational interviewing to improve weight loss in overweight and/or obese patients: A systematic review and meta-analysis of

randomized controlled trials. Obesity Reviews, 12(9), 709– 723.

Baskerville, N., Liddy, C., & Hogg, W. (2012). Systematic review and meta-analysis of practice facilitation within primary care settings. Annals of Family Medicine, 10(1), 63–74.

Bodenheimer, T. S. (2007). Building teams in primary care: Lessons from 15 case studies. Oakland, CA: California HealthCare Foundation.

Bordley, W. C., Margolis, P. A., Stuart, J., Lannon, C., & Keyes, L. (2001). Improving preventive service delivery through office systems. Pediatrics, 108(3), E41.

Centers for Disease Control and Prevention. (2008–2013). National Public Health Performance Standards Program. Local public health system performance assessment, version 2.0. Model Standards. Retrieved from http://www.cdc.gov/ nphpsp/documents/final-local-ms.pdf

Centers for Disease Control and Prevention. (2013). Obesity preva- lence: Minnesota. Retrieved from http://www.cdc.gov/dia- betes/atlas/countydata/obprev/data_Minnesota.pdf

Centers for Disease Control and Prevention. (2014). FastStats: Obesity and overweight. Retrieved from http://www.cdc. gov/nchs/fastats/obesity-overweight.htm

Community Benefit. (2014). Retrieved from http://www.naccho. org/topics/infrastructure/mapp/chahealthreform.cfm

Davis, D., O’Brien, M. A., Freemantle, N., Wolf, F. M., Mazmanian, P., & Taylor-Vaisey, A. (1999). Impact of formal contin- uing medical education: Do conferences, workshops, rounds, and other traditional continuing education activi- ties change physician behavior or health care outcomes? Journal of the American Medical Association, 282(9), 867– 874.

Derksen, R. E., Brink-Melis, W. J., Westerman, M. J., Dam, J. J., Seidell, J. C., & Visscher, T. L. (2012). A local consensus process making use of focus groups to enhance the imple-

TABLE 3. Intraclass Correlation for Omaha System Fol- low-up Outcome Ratings by Paired Raters: Administrator (n = 10), Clinician (n = 29), and Public Health Nurse Prac- tice Facilitator (n = 1)

Paired raters

ICC Omaha System KBS outcome ratings

Knowledge Behavior Status

Administrator—Clinician 0.53 0.46 0.35 PHN Practice facilitator—Administrator

0.63 0.46 0.31

PHN Practice facilitator—Clinician

0.53 0.46 0.44

Note. ICC = intraclass correlation; KBS = knowledge, behav- ior, and status; PHN = public health nurse.

PHN = Public Health Nurse

Knowledge Behavior Status

Av er

ag e

Ra tin

PHN - Baseline

PHN - Follow-up

Organization - Follow-up

Administration - Follow-up

Clinician - Follow-up

All - Follow-up

Figure 2. Average Baseline and Follow-up (33–39 months from baseline) Omaha System Outcome Ratings by Public Health Nurse Practice Facilitator (n = 1), Administrator (n = 10), Clinician (n = 29), and Organization (n = 10)

Source: Table 1 data.

230 Public Health Nursing Volume 32 Number 3 May/June 2015

mentation of a national integrated health care standard on obesity care. Family Practice, 29(1 Suppl.), i177–i184.

Felix, H., West, D., & Bursac, Z. (2008). Impact of USPSTF practice guidelines on clinician weight loss counseling as reported by obese patients. Preventive Medicine, 47(4), 394–397.

Finkelstein, E. A., Trogdon, J. G., Cohen, J.W., &Dietz,W. (2009). Fol- low-up medical spending attributable to obesity: Payer- and service-specificestimates.HealthAffairs,28(5),w822–w831.

Fitch,A., Everling, L., Fox, C., Goldberg, J.,Heim,C., Johnson,K., et al. (2013). Prevention and management of obesity for adults. UpdatedMay2013.InstituteforClinicalSystemsImprovement.

Glasgow, R. E., Goldstein, M. G., Ockene, J. K., & Pronk, N. P. (2004). Translating what we have learned into practice. Principles and hypotheses for interventions addressing mul- tiple Behaviors in primary care. American Journal of Pre- ventive Medicine, 27(2 Suppl.), 88–101.

Goldstein, M. G., Whitlock, E. P., & DePue, J. (2004). Multiple behavioral risk factor interventions in primary care. Sum- mary of research evidence. American Journal of Preventive Medicine, 27(2 Suppl.), 61–79.

Gowin, E., Avonts, D., Horst-Sikorska, W., Dytfeld, J., & Michalak, M. (2012). Stimulating preventive procedures in primary care. Effect of PIUPOZ program on the delivery of preventive procedures. Archives of Medical Science, 8(4), 704–710.

Hall, K., Gibbie, T., & Lubman, D. (2012). Motivational interviewing techniques – facilitating behaviour change in the general prac- tice setting.Australian Family Physician, 41(9), 660–667.

Health and Human Services. (2010). The surgeon general’s vision for a healthy and fit nation. Rockville, MD: HHS, Office of the Surgeon General.

Holtrop, J. S., Dosh, S. A., Torres, T., Arnold, A. K., Baumann, J., White, L. L., et al. (2009). Nurse consultation support to primary care practices to increase delivery of health Behav- ior services. Applied Nursing Research, 22(4), 243–249.

Institute of Medicine (IOM) (2012). Primary care and public health: Exploring integration to improve population health. Wash- ington, DC: National Academies Press.

Internal Revenue Service. (2013). REG-106499-12; Notice of pro- posed rulemaking community health needs assessments for charitable hospitals. Retrieved from http://www.irs.gov/ irb/2013-21_IRB/ar09.html#d0e3361

Jay, M., Schlair, S., Caldwell, R., Kalet, A., Sherman, S., & Gillespie, C. (2010). From the patient’s perspective: The impact of training on resident physician’s obesity counseling. Journal of General Internal Medicine, 25(5), 415–422.

Jensen, M. D., Ryan, D. H., Apovian, C. M., Ard, J. D., Comuzzie, A. G., Donato, K. A., et al. (2014). 2013 AHA/ACC/TOS guideline for the management of overweight and obesity in adults: A report of theAmericanCollege of Cardiology/AmericanHeart Associ- ation Task Force on Practice Guidelines and The Obesity Soci- ety.Journalof theAmericanCollegeofCardiology,63(25_PA), 2985–3023.doi:10.1016/j.jacc.2013.11.004.

Johnson, K. E., McMorris, B. J., Raynor, L. A., & Monsen, K. A. (2013). What big size you have! Using effect sizes to deter- mine the impact of public health nursing interventions. Applied Clinical Informatics, 4(3), 434–444.

Keller, L. O., Strohschein, S., Lia-Hoagberg, B., & Schaffer, M. A. (2004). Population-based public health interventions: Prac- tice-based and evidence-supported. Part I Public Health Nursing, 21(5), 453–468.

Koch, G. G. (1982). Intraclass correlation coefficient. In S. Kotz, & N. L. Johnson (Eds.), Encyclopedia of statistical sciences (Volume 4, pp. 213–217). New York: John Wiley and Sons.

Krist, A. H., Woolf, S. H., Frazier, C. O., Johnson, R. E., Rothemich, S. F., Wilson, D. B., et al. (2008). An electronic linkage sys- tem for health behavior counseling: Effect on delivery of

the 5 A’s. American Journal of Preventive Medicine, 35(5 Suppl.), S350–S358.

Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174.

Martin, K. S. (2005). The Omaha System: A key to practice, docu- mentation, and information management (2nd ed.). Omaha, NE: Health Connections Press.

Monsen, K. A., Attleson, I. S., Erickson, K. J., Neely, C., Oftedahl, G., & Thorson, D. R. (2014). Translation of obesity practice guidelines: Interprofessional perspectives regarding the impact of public health nurse system-level intervention. Public Health Nursing, Jul 13. doi:10.1111/phn.12139. [Epub ahead of print]

National Heart, Lung, and Blood Institute. (1998). Clinical guide- lines on the identification, evaluation, and treatment of overweight and obesity in adults: The evidence report. Retrieved from http://www.nhlbi.nih.gov/files/docs/guide- lines/ob_gdlns.pdf

National Institute of Health. (2000). The practical guide: Identifica- tion, evaluation, and treatment of overweight and obesity in adults. Retrieved from http://www.nhlbi.nih.gov/files/docs/ guidelines/prctgd_c.pdf

Ogden, C. L., Carroll, M. D., Kit, B. K., & Flegal, K. M. (2012). Prev- alence of obesity in the United States, 2009–2010. NCHS data brief, no 82. Hyattsville, MD: National Center for Health Statistics.

Pimlott, N. (2008). Who has time for family medicine? Canadian Family Physician, 54(1), 14–16.

Ploeg, J., Davies, B., Edwards, N., Gifford, W., & Miller, P. E. (2007). Factors influencing best-practice guideline imple- mentation: Lessons learned from administrators, nursing staff, and project leaders. Worldviews on Evidence-Based Nursing, 4(4), 210–219.

Poobalan, A. S., Aucott, L. S., Ahmed, S., & Smith, W. C. (2010). Analysis of the UK recommendations on obesity based on a proposed implementation framework. Biomed Central Pub- lic Health, 10(1), 17.

Schriefer, S. P., Landis, S. E., Turbow, D. J., & Patch, S. C. (2009). Effect of a computerized body mass index prompt on diag- nosis and treatment of adult obesity. Family Medicine, 41 (7), 502–507.

Schuster, R. J., Tasosa, J., & Terwoord, N. A. (2008). Translational research–implementation of NHLBI obesity guidelines in a primary care community setting: The physician obesity awareness project. The Journal of Nutrition, Health Aging, 12(10), 764S–769S.

Strategies to Overcome and Prevent Obesity (STOP) Alliance. (2010). Research and Policy, Research Center, Survey Results. Provider/patient survey on obesity in primary care setting. Retrieved from http://www.stopobesityalliance.org/ research-andpolicy/research-center/survey-results

Thorson, D. R., Erickson, K. J., Attleson, I. S., & Monsen, K. A. (2014). Transforming an evidenced-based adult obesity guideline into clinical practice. Otter Tail County Public Health (MN). Retrieved from https://dl.dropboxusercon- tent.com/u/16553143/m/PS4H.pdf

United StatesPreventive ServicesTaskForce. (2012). Screening for and management of obesity in adults: U.S. preventive services task force recommendation statement. AHRQ Publication No. 11- 05159-EF-2. Retrieved from http://www.uspreventiveservic- estaskforce.org/uspstf11/obeseadult/obesers.htm

Zhu, D. Q., Norman, I. J., & While, A. E. (2011). The relationship between doctors’ and nurses’ own weight status and their weight management practices: A systematic review. Obesity Reviews, 12(6), 459–469.

Erickson et al.: Translation of Obesity Practice Guidelines: Measurement 231