Week 2: Healthcare Systems Leadership

Formative and Summative Evaluation Sample

The other focus in in Week 2 of this course is articulating your formative evaluation you will use throughout your project implementation to check your intervention fidelity. 

Why are we using a formative and a summative evaluation? Why don't we just measure the effects of our intervention and be done with the evaluation process? >> Our DNP project leader told us that we need to conduct an audit to verify the intervention is correctly implemented. What was that called when the intervention is implemented exactly as discussed in the evidence? >> This is called intervention fidelity. The intervention fidelity is high when there is little variation. If there is a wide range of variation to how the nurses implement, the intervention fidelity is low. In other words, if the intervention fidelity is high, we are implementing the intervention as it is described in the evidence. So we want to ensure the fidelity is high. On the evening shift, the staff want to cut some corners because there is no medical receptionist. And we are having trouble scheduling patient care technicians to help with hourly rounding and vital signs. I'm concerned there is a wide variation in implementing hourly rounding on the evening shift. >> On the night shift, I am asking the staff to implement our rounding intervention exactly as we were educated. At the end of each shift, we have a huddle to audit our implementation of the hourly rounding intervention. We audit if the staff were unable to conduct hourly rounding. We discuss the barriers and figure out solutions. Our team is committed to doing this right and ensuring intervention fidelity on the night shift. >> During the day, we have many interruptions, I decided to use the formative evaluation checklist to audit each staff nurse, because there is a variation in the hourly rounding implementation between nurses. I discussed variations with each nurse, so that each nurse understands how to correctly implement hourly rounding. >> Yes, using the checklist audit each nurses hourly rounding is a better way to conduct the formative evaluation. You are assessing the implementation fidelity for all nurses during our day shift. It hourly rounding is implemented in the same way by every nurse. There should be less variation in the outcomes. >> So I should use the checklist to audit each nurses hourly rounding instead of auditing all nurses together for the shift. I see how a variation in hourly rounding for each nurse might affect our outcome. I will speak to the staff and we can start auditing each nurse tonight. >> The DNP project leader said he will measure patient and nurse satisfaction rates at the end of the project implementation for the summative evaluation. >> Yes, I will be interested to know if nurse and patient satisfaction are improved during this implementation. >> So just to summarize my understanding here, the formative evaluation is a frequent audit conducted throughout the implementation to check the intervention fidelity. The formative evaluation frequently identifies variations in intervention fidelity, and we work to find solutions to those variations. The summative evaluation measures the effect of the intervention on the outcomes. So in this case, patient and nurse satisfaction were measured before the hourly rounding intervention was implemented and will be measured again after. We're hoping that there is a significant difference in patient or nurse satisfaction after the implementation of hourly rounding. >> Yes, I am really hoping our hard work results in increased nurse and patient satisfaction.