Integrated evaluation project

Week 10: Knowing what works. Evaluation for public health

FINAL ASSIGNMENT: INTEGRATED EVALUATION PLAN

D R T E R E S A B U R G E S S

University of Adelaide 2

We acknowledge and pay our respects to the Kaurna people, the traditional custodians whose ancestral lands we gather on.

We acknowledge the deep feelings of attachment and relationship of the Kaurna people to country and we respect and value their past, present and ongoing connection to the land and cultural beliefs.

Integrated evaluation plan (45%)

The program you will develop an evaluation plan for is a practice nurse-led lifestyle program to support weight loss.

It is delivered within 2 general practices in the City of Playford, South Australia:

• GP Plus Health Care Centre Elizabeth

• Unihealth Playford GP Super Clinic

The justification for the program is:

• Obesity is associated with an increased risk of a wide range of conditions

• Australia has one of the highest rates of overweight and obesity

• Even a small amount of weight loss (e.g., 5%) can be beneficial

The Program Goal:

• To support participants to make a sustained reduction of at least 5% of their body weight, over a two-year time period, with the ultimate goal of reducing weight- related morbidity and mortality in the Australian population

University of Adelaide 3

Integrated evaluation plan (45%)

Program outline

• The program includes key behavioural strategies including self-monitoring, lapse management and goal setting, delivered in non-judgemental, collaborative and empathetic consultations.

The program content includes:

• Six fortnightly education and support sessions

• Three follow-up support sessions (at 6, 9, 12 months)

• Training, support, and mentoring for practice nurses (from an experienced dietician)

The target population is:

• Adults, 18-75 years old, with a Body Mass Index ≥ 30 who attend the 2 nominated general practices

University of Adelaide 4

Integrated evaluation plan (45%)

Your assignment is to develop and report a detailed evaluation plan for this program, using an integrated approach, drawing on mixed methods and your plan must include economic evaluation

It should be 2000 words, plus appendices

Due date: Sunday 3rd November

University of Adelaide

5

Integrated evaluation plan (45%)

Report structure should follow steps for evaluation planning (Round et al 2005)

1. Describe the program (including SMART objectives)

2. Evaluation preview (including engaging stakeholders, purpose and type of evaluation, key evaluation questions)

3. Focus the evaluation design

• Evaluation design should include two components:

I. An evaluation planning process grid

II. A proposed health economic evaluation

4. Collect data (including methods and sample questions)

5. Analyse and interpret data (consider appropriate types of analysis)

6. Disseminate lessons learnt (how, and to whom)

University of Adelaide 6

Six Steps for Evaluation Planning

Victorian Department of Health and Human Services

1. Describe the program

2. Evaluation preview

3. Focus the evaluation design

4. Collect data

5. Analyse and interpret data

6. Disseminate lessons learnt

University of Adelaide 7

Round et al, 2005

Step 1: Describe the program

Step 3: Focus evaluation design

Step 4: Collect data

Step 5: Analyse and interpret data

Step 6: Disseminate lessons learnt

Step 2: Evaluation preview

SMART objectives

• Specific – what do you want to achieve, by when, for who, where?

• Measurable – can you measure the change?

• Achievable – is your goal reached with a reasonable amount of effort?

• Realistic – is it realistically possible using the skills and resources you have available?

• Time limited – can you reach your goal in the allotted time?

University of Adelaide 8

Writing SMART objectives

• Objectives describe the steps you will undertake to help you to meet the program goal - so what do you think would be required to help to ensure a sustained reduction of at least 5% of their body weight, over a two-year time period?

o Consider:

‒ the project activities

‒ the available timelines – don’t be afraid to be ambitious as well as realistic

‒ How measurable your objectives are

• Always test every objective to yourself: will this objective contribute to the overall goal? How and why? If you can answer that clearly then your objective should be valid.

• Be very specific – e.g. not “participants will be encouraged to increase vegetable consumption by …” say “75% of participants will increase…”

• There are 2 groups involved in the program

• Remember you have 2 years, so you might have some 1 year objectives?

• Your Evaluation Planning Process Grid will help you to identify your questions around process and impact

University of Adelaide 9

Evaluation planning process grid

Summarise and present the evaluation planning process

For each program objective

• What are they key questions to be asked?

• What information do we need to answer these questions?

• How will this information be collected?

• What is the budget necessary to undertake the data collection?

Also includes key questions relating to the overall project, and planning space for preparation and dissemination of the final report

University of Adelaide 10

Round et al, 2005.

University of Adelaide 11

Evaluation planning process grid

Round et al 2005

Australia’s Health 2018

https://www.aihw.gov.au/reports/australias-health/australias-health- 2018/contents/table-of-contents

City of Playford Profile

https://www.playford.sa.gov.au/council/about-council/city-profile

GP Plus Health Care Centre Elizabeth

https://www.sahealth.sa.gov.au/wps/wcm/connect/public+content/sa+health+intern et/health+services/gp+plus+health+care+services+and+centres/elizabeth+gp+plus+h ealth+care+centre/gp+plus+health+care+centre+elizabeth

Unihealth Playford SuperClinic

https://www.adelaideunicare.com.au/index.php/our-practices/unihealth-playford

University of Adelaide 12