SOCW 6311 Week 9 - Assignment: Drafting a Process Evaluation

Week 9: Process Evaluation

Process evaluation ensures that a program is implemented as intended—which is critical for successful adoption of evidence-based practice. Process evaluation can help explain why there are differences among outcomes in programs or practice models. They might reveal difference in the education and training of providers of the service, in characteristics of the participants in the program, or in length and frequency of therapy sessions, among other things.

Because a large proportion of process evaluation utilizes qualitative research, you examine the process of qualitative data analysis and the reporting of results of such analysis, you apply process evaluation reporting techniques to improve upon a qualitative research report, and you create a process evaluation plan.

Learning Objectives

Students will:

· Compare process evaluation reports

· Generate a process evaluation plan

Learning Resources

Required Readings

Dudley, J. R. (2020). Social work evaluation: Enhancing what we do (3rd ed.) Oxford University Press.

· Chapter 8, “Improving How Programs and Practice Work” (pp. 171–208)

Becker, L. A. (1999). Statistical and clinical significance. Retrieved from https://www.uccs.edu/lbecker/clinsig

Man-Son-Hing, M., Laupacis, A., O’Rourke, K., Molnar, F. J., Mahon, J., Chan, K. B., & Wells, G. (2002). Determination of the clinical importance of study results. Journal of general internal medicine, 17(6), 469-476. 

Plummer, S.-B., Makris, S., & Brocksen S. (Eds.). (2014b). Social work case studies: Concentration year. Baltimore, MD: Laureate International Universities Publishing. [Vital Source e-reader].

Read the following section:

· “Social Work Research: Qualitative Groups” (pp. 68–69)

Document: Bliss, M. J., & Emshoff, J. G. (2002). Workbook for designing a process evaluation. Retrieved from http://beta.roadsafetyevaluation.com/evaluationguides/info/workbook-for-designing-a-process-evaluation.pdf (PDF)

Georgia Department of Human Resources, Division of Public Health.

Optional Resources

QSR International. (n. d.). NVivo 10. Retrieved October 17, 2013, from http://www.qsrinternational.com/products_nvivo.aspx

Use this webpage to view a demonstration of how qualitative data analysis can be assisted by software. You may explore any of the demos, but it is recommended that you start with NVivo eDemo. In order to view this demo, you will need to register, and download (or enable) the latest Adobe Flash Player.

Examples of Process Evaluation

Boyce, C., & Neale, P. (2006). Conducting in-depth interviews: A guide for designing and conducting in-depth interviews for evaluation input. Pathfinder International Tool Series: Monitoring and Evaluation – 2. Retrieved from http://www.cpc.unc.edu/measure/training/materials/data-quality-portuguese/m_e_tool_series_indepth_interviews.pdf

Hesselink, A. E., & Harting, J. (2011). Process evaluation of a multiple risk factor perinatal programme for a hard-to-reach minority group. Journal of Advanced Nursing, 67(9), 2026–2037.

Note: Retrieved from the Walden Library databases.

Lee, E., Esaki, N., & Greene, R. (2009). Collocation: Integrating child welfare and substance abuse services. Journal of Social Work Practice in the Addictions, 9(1), 55–70.

Note: Retrieved from the Walden Library databases.

Maxwell, N., Scourfield, J., Holland, S., Featherstone, B., & Lee, J. (2012). The benefits and challenges of training child protection social workers in father engagement. Child Abuse Review, 21(4), 299–310.

Note: Retrieved from the Walden Library databases.

Assignment: Drafting a Process Evaluation

The steps for process evaluation outlined by Bliss and Emshoff (2002) may seem very similar to those for conducting other types of evaluation that you have learned about in this course; in fact, it is the purpose and timing of a process evaluation that most distinguish it from other types of evaluation. A process evaluation is conducted during the implementation of the program to evaluate whether the program has been implemented as intended and how the delivery of a program can be improved. A process evaluation can also be useful in supporting an outcome evaluation by helping to determine the reason behind program outcomes.

There are several reasons for conducting process evaluation throughout the implementation of a program. Chief among them is to compare the program that is being delivered to the original program plan, in order to identify gaps and make improvements. Therefore, documentation from the planning stage may prove useful when planning a process evaluation.

For this Assignment, you either build on the work that you completed in Weeks 6, 7, and 8 related to a support group for caregivers, or on your knowledge about a program with which you are familiar. Review the resource “Workbook for Designing a Process Evaluation”.

Provide a 4-page plan for a process evaluation. Include the following minimal information:

· A description of the key program elements

· A description of the strategies that the program uses to produce change

· A description of the needs of the target population

· An explanation of why a process evaluation is important for the program

· A plan for building relationships with the staff and management

· Broad questions to be answered by the process evaluation

· Specific questions to be answered by the process evaluation

· A plan for gathering and analyzing the information

Must include at least 4 references and citations using the weekly reading materials and the references used in the original papers.

Week 6:

Developing Project Evaluations

Stakeholder Analysis

The main stakeholders in the program development for the agency transport system are important as they help in the daily activities, and implementation of the program will benefit them of the greater deal in the program evaluation process (Dudley, 2014). The transport evaluation process is to help patients in the mental health program treated in an Intensive Outpatient Program (IOP) to move in scheduled appointments without feeling rushed to complete their sessions or worried about how they can reach their residential areas.

The main stakeholder is the evaluators, who are the people tasked with evaluating the problem and the causal action needed. The second stakeholders are the mental health patients, who are the primary factor in consideration for the recommendation for the program evaluation. The third stakeholders are the counselling staff, who feel rushed to complete their sessions and see their patients less benefiting from the program due to the lack of proper scheduling and coordination among the group. The transport company and its drivers are also stakeholders because they face pressure from the patients and the staff providers who nd help develop the best care possible. The other stakeholders are the mental healthcare agency that takes their patients to the IPO every morning and would not like to be distracted by scheduling options, driving patients back to their residents. The other greatest stakeholders include the funders who pay for the program to ensure that patients get the best care possible and develop a strong connection for operational funding programs.

Thus, a proper stakeholder analysis makes it possible to answer all the concerns they might have about accountability and use of resources, making the overall outcome of the evaluation passible and updates in the right manner.

Program Evaluation Plan

The purpose of the evaluation is to determine the best way possible the scheduling of the patients’ needs to be done for transportation back to their sober residentials from the outpatient counselling services. The evaluation seeks to identify how the scheduling plan will benefit the counselling services and staff, the patients, and the transport system to make the aim of the counselling services beneficial. In dealing with mental health patients, it is not a question of reason but how to make concentration more adequate without worrying about how to go home and using appointments. Thus, the evaluation will assess if the investment is worth it or not make any change in how the mental health patients feel when others have completed their sessions and appointments, and it is their turn to get help (Science, 2017).

The evaluation will address the effectiveness of the existing transport arrangement and the good for scheduling, to the stakeholder's concerns, primarily the Outpatient services, the drivers, and the healthcare leadership aspect. The questions that will be addressed in the evaluation will also include the assessment process which impact the healthcare community and the surrounding members in the clinic who receive different services and neighbors, on how they perceive the program. The perception of all the staff members towards the program is meant to benefit them and be the people on the ground. They might have different ideas and opinions that can serve to be more beneficial to the general outcome. The last question that will be addressed in the evaluation process is the lessons learnt in the implementation process for future reference when helping patients with mental health concerning transportation issues (Science, 2017).

In the same evaluation, outcomes will be evaluated based on the community changes to the effect of intervention in the transportation system, in terms of knowledge about the program's existence and how it helped. It will also be vital to answering programmatic and policy concern changes that made the entire process a success. In conclusion, questions about if the changes in the scheduling process helped improve the outcomes of healthcare and the patient's concentration will be vital (Science, 2017).

The concerns of the stakeholders in the implementation of the process range on the question of how the scheduling will be done so that it does not lead to more complications (Logan & Royse, 2010). For instance, will some patients feel more prioritized than others for taking different schedule times and thus create unease in adopting and adhering to the schedule? On the contrary, will communicating the changes in the operations become difficult to the patients, which will create unrest in their sober residential areas? Will the staff and the entire organization and counselling services demonstrate cooperation with the system? Will the funding sector feel the development is worth consideration and impactful to the general outcome of the organization? The evaluation can be addressed by taking a keen interest in the records and making all viable solutions and explanations available, merit accountability and detail to work (Science, 2017).

Evaluation of a program and stakeholder concerns is the best way to keep all parties in the organization and present the findings in the best way possible, considering all the interests of the people involved and making the right decisions.

References

Dudley, J. R. (2014). Social work evaluation: Enhancing what we do. (2nd ed.) Chicago, IL: Lyceum Book

Logan, T. K., & Royse, D. (2010). Program evaluation studies. In B. Thyer (Ed.), The handbook of social work research methods (2nd ed., pp. 221–240)

Science, C. (2017, November 1). The Step-by-Step Guide to Evaluation: How to Become Savvy Evaluation Consumers. W.K. Kellogg Foundation. https://www.wkkf.org/resource-directory/resources/2017/11/the-step-by-step-guide-to-evaluation--how-to-become-savvy-evaluation-consumers.

Week 7:

Week Seven - Developing a Logic Model Outline

A logic model is a picture representation that shows the association between activities, outputs, resources, and the effects of a particular program (Randolph and Thyer 2010). The following tables and details are compiled to address the requirements of the week seven assignment. The following details will provide a practice-level logic model, a program-level logic model to address the needs of Helen in the Petrakis case history. The content below will also include elaboration on the practice-level logic model outline and the program-level logic model.

Practice-Level Logic Model Outline

Problem	Needs	Underlying Causes	Intervention Activities	Outcomes
Hellen mainly has management issues with taking care of Magda and her role as a caregiver at home, which brings her anxiety, stress and overwhelmed that she is doing it alone. She has back pains. Hellen feels failed as a mother and wife as John is dissatisfied with food at home and his son lacks clean laundry. Hellen lacks time out shopping and eating with friends. Alec Hellen’s son has drug abuse and steals from Magda’s house, and Hellen cannot tell John about him to avoid him being humiliated. Helen’s health is at stake too.	Hellen needs to address self-time management in care for Magda, her role as mother and wife too. She needs to know how to manage her anxiety, too; she needs support from the family to address the issue in the family. Hellen also needs to understand how to manage Magda’s care with the financial constrain the family has	The causes for Hellen’s problem are mainly on lack of family support in helping Magda. Hellen feels not wife or mother enough due to a lack of appreciation for her efforts. Alec adds trouble by stealing medicine, jewels and checks to support his drug addiction. Hellen seems to be overthinking, which gives him lots of pressure leading to her illness.	The best intervention is to use psychotherapy techniques to address anxiety, such as the cognitive restructuring approach to control the outcomes and her perception, stress and pressure. Counselling will offer positive gain, mainly on helping Hellen to cope with medication while at the same time reducing things she cannot control, which contributes to stress. Physical therapy can help with acetaminophen to relieve any pressure.	The outcomes when all help in care for Magda will reduce Hellen's anxiety and stress since her role will not be diverse. Good management will help each other appreciate the outcomes both financially, emotionally. The result could include seeking help for Alec’s addition and looking for a Living arrangement for Magda as a family.

Program-Level Logic Model Outline

Problem	Needs	Underlying Causes	Intervention Activities	Outcomes
Hellen’s problems are stress, anxiety, health management for her stress-causing condition, the support from the family at large, mainly for home chores and taking care of the family. Ale’s substance issue. Magna needs a housing arrangement.	The primary needs are dealing with anxiety, housing plan for Magda, Supportive family plan and delegation of duties, psychiatric evaluations.	The leading cause is the lack of a supportive family. Hellen has lost of responsibilities. Hellen feels less of a mother and wife due to a lack of recognition for all she does. Alec, instead of helping, adds more stress by stealing medicine, cheques and jewellery. There is no housing arrangement for Magda.	The solution-based goal-directed intervention will help mainly in targeting the solution to the said problems. Interventions to focus on management, resilience, resources are manageable, Personal interventions will be necessary, and family therapy to help even the children and John accept the help from children. Psychodynamic therapy can help the whole family through family counselling sessions.	Hellen will need Short Term: individual psychodynamic counselling therapy sessions for like 13 sessions to help solve the short time issues affecting him to cope with stress and anxiety. Long term: Counselling sessions for a year could be valuable for helping the family to deal with stressors that come with family dynamics and helping their mother to reduce her stress levels.

Practice-Level Logic Model

Hellen has a problem with management challenges which pushes her to stress and depression. She wants to fit perfectly in her life when her in-law was not sick and, at the same time, help her out with her needs. The situation pushes her to the point she feels a failure as a mother and wife. Her interventions need to include management issues and stressors in life and overall interventions (Dudley, 2020). 

The best way to deal with Helen’s situation is using cognitive restructuring to change her perception of stress and embrace positive thoughts of positivity (Ezegbe et al., 2018). The condition can help Helen to act normal even when distressed and reduce the stimulation of nervousness. Thus, even if she can get anxious from the situations she is facing, it can help maintain positive thoughts and remain more productive.

Program-Level Logic Model

The Petrakis family's problem is not individual, although it revolves around Hellen, who is the centre of it all in the analysis. Thus, closed family therapy can be sued in the psychodynamic therapy approach to involve the family and cultural aspects in taking care of their own without feeling like they apply strangers as Hellen felt. The main aim should be to share responsibility in consideration for Hellen’s elevation of stress and anxiety. It should be solution based in also finding solid help for Magda and Alec, who has a drugs problem (Turpin, 2019).

Short term: Hellen needs individual therapy for at least 13 sessions to help her develop a positive perception in dealing with anxieties.

Long term: Counselling sessions for a year could be valuable for helping the family to deal with stressors that come with family dynamics and helping their mother to reduce her stress levels. A housing plan is required, which can be supported by the entire family than just Jon and Hellen using their vacation savings. Scale intervals can explain how Helen feels and the interventions needed.

References

Dudley, J. R. (2020). Social work evaluation: Enhancing what we do. Oxford University Press, USA.

Ezegbe, B. N., Ede, M. O., Eseadi, C., Nwaubani, O. O., Akaneme, I. N., Aye, E. N., ... & Ugwu, U. (2018). Effect of music therapy combined with cognitive restructuring therapy on emotional distress in a sample of Nigerian married couples. Medicine, 97(34).

Randolph, K. A., & Thyer, B. (2010). Logic models. The Handbook of Social Work Methods, 547-561.

Turpin, C. (2019). COGNITIVE ANALYTIC THERAPY AND PSYCHODYNAMIC INTERPERSONAL THERAPY. The Handbook of Brief Therapies: A Practical Guide, 113.

Week 8:

The resources needed to operate this service

To complete the needs assessment on the targeted population, a social worker requires several resources to facilitate the process. Due to the facilitation during the collection of primary data, financial resources are needed. The resources will be used in traveling and other expenses during data collection. Further, questionnaires will be used in data collection (Tutty & Rothery, 2010). The social worker will source the questionnaires and print them prior to collecting primary data. Third, secondary sources of data such as hospital reports and vital statistics will be used. Finally, a team will be required where roles will be distributed to cover a wider area and topic.

The program activities

The assessment will be completed using five program activities. The first activity is planning and designing the assessment. In this process, the scope and objectives have already been established. Therefore, the first process will involve coordination with stakeholders, giving a definition of information needs, planning all the logistics and operations during data collection, designing the methodology, designing all the tools and resources needed in data collection, and organizing the analysis. The second activity is implementation where secondary data will be reviewed and primary data will be collected (Dudley, 2020). Thirdly, the social worker will conduct a clean and process activity. Here, both the primary and secondary data will be entered, cleaned, and processed. The fourth activity will be data analysis. The data will be interpreted and validated. Finally, the last activity will be sharing of data through the assessment report.

The desired outcomes

The assessment seeks to achieve all the set objectives. The social worker expects cooperation from participants during the collection of data. The assessment will identify the challenges faced by victims of domestic violence in seeking justice. Respondents from key informants will indicate the problems domestic violence victims face when interacting with law enforcement officers. Further, the assessment report will show the reasons why victims of domestic violence choose to report or not report (Dudley, 2020). The average number of cases reported will be indicated against the current measures to curb domestic violence. What’s more, the report will indicate the role played by other stakeholders in the community such as physicians, religious leaders, and policymakers. In the assessment report, the primary data will be compared to secondary literature.

A plan for gathering information about the population served

The report will indicate a needs assessment for victims of domestic violence. The data will be sourced from victims of domestic violence, caregivers, individuals who interacted with the recipients of domestic violence, religious leaders, and law enforcement officers. Information will be collected through questionnaires. In this case, the data will be collected through online and offline questionnaires (Tutty & Rothery, 2010). After obtaining consent from the participants, they will be required to select the most convenient method of answering questions. Since the data collection will take several days, the teams will be debriefed on regular basis to address any potential issues. Participants will be identified from police reports and other secondary sources.

Justifications for your plans and decisions

The needs assessment plan is a complex process that involved a vulnerable group. Therefore, the designed plans and decisions are meant to ensure maximum results. A five activity plan was used to exhaust all essential elements in the assessment. Each step is key in data collection, implementation, and analysis. Further, several key informants will participate in the assessment. A high number of participants increase the validity and reliability of results. Finally, the assessment used both primary and secondary data. Using the two sets of data will allow the social workers to compare data. As such, the assessment report will contain the best needs report for the population served.

Conclusion

During the implementation stage, it is important to review the process by making a follow-up. The implementation stage is core to the entire process since it involves the collection of primary data and reviewing secondary data. Therefore, the assessment team will discuss the collection of data and possible problems on regular basis. A follow-up will be conducted to ensure that team members understand the questions they are to ask. Feedback collection will be a continuous process to allow easier data validation. In this case, follow-up will be achieved through debriefing which will be conducted after the completion of data collection.

References

Dudley, J. R. (2020). Social work evaluation: Enhancing what we do (3rd ed.) Oxford University Press.

Tutty, L. M., & Rothery, M. A. (2010). Needs assessments. In B. Thyer (Ed.), The handbook of social work research methods (2nd ed., pp. 149–162). Thousand Oaks, CA: Sage. (PDF)