help 8764
mordyggggg
cf_sample_presentation02.pptxResults of Evaluation of United Ways of Northeast Florida 2-1-1[Project].
Capella Learner
Capella University
PSY8763 – Program Evaluation
Instructor
Introduction
Information and referral service
Northeast Florida service area
All residents
Established in 2000
United Ways of Northeast Florida (UWNEF) 2-1-1 is an information and referral for community health and social service agencies. The range of services include what we call the “big three” needs: shelter, food, and utilities; however, legal, psychiatric, medical, and other social service needs are also addressed.
We serve the nine counties of northeast Florida that include Baker, Clay, Columbia, Duval, Hamilton, Nassau, Putnam, Saint Johns, and Suwannee counties.
All residents may contact UWNEFL 2-1-1 by phone, SMS, Web chat, or email.
The UWNEFL was originally established in 2000 by the United Way of Northeast Florida, based in Jacksonville, but now supports three separate United Way agencies to include the original United Way of Northeast Florida as well as Saint John’s United Way and the Suwannee Valley United Way.
2
Introduction
Contact Methods
Phone
SMS
The accrediting body, the Alliance of Information and Referral (AIRS), requires each information and referral (I&R) provider to ensure their database resources are tailored to meet the needs of their service area (n.d.).
Additionally, AIRS (n.d.) requires knowledge of the communities' demographics served to include but not limited to race and economic status.
AIRS (n.d.) further requires that all standards are maintained throughout the life of the accreditation period.
The State of Florida Statute 408.918 (Florida Legislature, 2020) requires all I&R agencies to maintain accreditation with the Alliance of Information and Referral Services to operate within the State as a 2-1-1 organization lawfully.
The database was discovered to have been largely neglected; some resources were four to five years out of date and required updating.
Due to the low number of resources currently listed in the database in the outlying counties, it was suspected there were needs not being addressed.
According to Leider et al. (2020), rural areas often have fewer resources to address social determinants of health discrepancies, including “housing, education, [and] access to healthy foods” (p. 1285). The initial review by the AIRS accreditation staff these findings were confirmed, and the center was provided six months to undertake improvements.
3
Introduction
Program goals and objectives
Updated information and resources
Meaningful referrals
Tailored to community needs
The accrediting body, the Alliance of Information and Referral (AIRS), requires each information and referral (I&R) provider to ensure their database resources are tailored to meet the needs of their service area (n.d.).
Additionally, AIRS (n.d.) requires knowledge of the communities' demographics served to include but not limited to race and economic status.
AIRS (n.d.) further requires that all standards are maintained throughout the life of the accreditation period.
The State of Florida Statute 408.918 (Florida Legislature, 2020) requires all I&R agencies to obtain and maintain accreditation with the Alliance of Information and Referral Services to operate within the State as a 2-1-1 organization lawfully.
The database was discovered to have been largely neglected; some resources were four to five years out of date and required updating.
Due to the low number of resources currently listed in the database in the outlying counties it was suspected there were needs not being addressed.
According to Leider et al. (2020), rural areas often have fewer resources to address social determinants of health discrepancies, including “housing, education, [and] access to healthy foods” (p. 1285). The initial review by the AIRS accreditation staff these findings were confirmed, and the center was provided six months to undertake improvements.
4
Introduction
Significant evaluation findings
Limited database resources
Client dissatisfaction
New resources needed
Limited resources were available in Baker, Clay, Columbia, Hamilton, Nassau, Putnam, Saint Johns, and Suwannee counties.
Clients in outlying counties were unsatisfied with the lack of resources.
Existing resources must be identified and added to the database.
5
Introduction
Evaluation Methods
Interviews
Surveys
Secondary sources
Interviews were initially conducted with clients contacting UWNEFL 2-1-1 from outlying counties and their responses were used to develop a written survey.
The written surveys were offered to all clients from the outlying counties to gather information related to the referrals provided to their community agencies.
Secondary sources (Royse et al., 2016) were reviewed to include United States Census demographic information for the eight outlying counties. Additional secondary source included the State of Florida Health Department to identify health and social service needs identified within those counties.
The Joint Committee on Standards for Educational Evaluation (JCSEE) were used to conduct the program evaluation, as they are recognized by many program evaluators such as Ruhe and Boudreau (2013). The standards assure the assessment is feasible, meets industry standards, is done accurately, and the process is transparent (Ruhe & Bourdreau, 2013).
6
Methods
Data and Collection
Interviews
Surveys
Demographics
Needs
Data and Collection
Interviews were conducted of 10% of the clients from outlying eight counties that contacted the UNWNEFL 2-1-1.
The interviews were open ended to gather client specific issues related to referrals they received as their perspectives were extremely important as the primary consumers served by the center.
The questions covered several issues including their satisfaction with the center referral process, the overall availability of services, discussion of the eligibility criteria, and satisfaction with the services received at the community agency.
The UWNEFL 2-1-1 staff conducted the interviews through telephone contacts.
The results were recorded by staff in writing and the calls were electronically recorded and transcribed into text using the UWNEFL 2-1-1 analytics platform.
The interviews were used to produce categories around topics held in common by the clients and converted into written surveys.
Surveys developed from the iterative interview process were then offered to all clients from the outlying counties in October and November of 2020.
United States Census data provided poverty, education, and employment status rates for both the county and neighborhoods.
State of Florida Health Department reports were used to identify specific needs around health, crime, drug use, and other social needs for each county.
7
Methods
Methodological Challenges
Difficulties
Limitations
Methodological Challenges:
Difficulties encountered included low response rates of initial telephone interviews. To obtain a 10 percent sample of responses, approximately 25 percent of clients needed to be called due to many calls going unanswered or clients that declined to participate.
The response rate of written responses completed accounted for 16 percent of all clients from the outlying counties despite all clients being offered the opportunity to participate.
Due to the low response rates and the low percentage of callers from each county, total needs are difficult to identify. Florida Department of Health data provided a more accurate identification of needs, which was applied to the population numbers gathered from the Census.
Low responses rates also make it difficult to draw firm conclusions from the data, as dissatisfied callers may have responded to the survey. Therefore, only broad correlations and conclusions are made.
8
Findings
61% client satisfaction with UWNEFL 2-1-1 referral process
85% dissatisfaction with resource availability
93% of clients dissatisfied when services not provided
100% client satisfaction when services are provided
Overall satisfaction rate of 61 percent for the process used to refer clients to community resources.
85 percent of clients were dissatisfied with the availability of resources within their community.
When clients were not served by the agency, 93 percent were dissatisfied with the resource.
All clients who were successfully linked to resources were satisfied with the outcome.
9
Interpretation of Results
Customer satisfaction tied to resource availability
Lack of available resources cited often
Findings align with Leider et al. (2020), that rural areas often have fewer resources to address social determinants of health discrepancies.
However, due to the low survey responses, firm conclusions have been avoided.
Despite the low response, there is a strong link between the availability of resources and the level of satisfaction received.
Clients overwhelming identified dissatisfaction with the resources being provided within the outlying service area counties.
10
Recommendations
Critical to identify additional resources
Provide more information
Additional problem-solving
Share content with communities
It is recommended that additional resources be identified within the outlying counties by:
reaching out to the points of contact with current community resources and, using the snowball technique (Royse et al., 2016), ask the providers for the names of other services within the county.
Utilize the Internal Revenue Service listing of nonprofit agencies and attempt to identify the services they provide.
Travel to each of the counties and visit social service agencies not currently contained in the database.
To increase customer satisfaction, it is recommended that staff provide additional information and context around the limitation of resources within the communities.
Until new resources can be identified staff are advised to problem solve with clients to address other needs in lieu of the identified need to free up client resources to address the unmet need. For example, food resources have been identified in all counties and providing referrals for food may permit the clients to use the funds to address rent or utility needs.
The final recommendation is to conduct town halls within the communities in coordination with private and public service providers and share finding of evaluation to identify new resources and suggest the creation of new services with new partner agencies and addition of new services with existing agencies. The engagement or the clients and community stakeholders in the process will aid in their ownership of the issue and help gain their cooperation (Holden & Zimmerman, 2008).
11
Conclusion
Customers are dissatisfied
Resources are needed
2-1-1 accreditation
Florida statute
Customers frequently cite the lack of resources within the outlying counties as a source of dissatisfaction with the UWNEFL 2-1-1 program.
It is critical to identify new resources to meet the needs of the rural counties around the social determinants of health.
The accreditation of the UWNEFL 2-1-1 requires the identification and maintenance of resources that are tailored to each community. There is an expectation from AIRS that new resources will be added for the rural counties.
It is required by Florida Statute that any center serving as a 2-1-1 meet and maintain the accreditation standards of AIRS.
12
References
Alliance of Information and Referral Systems. (n.d.). AIRS Resource Database Access Options. https://www.airs.org/i4a/pages/index.cfm?pageid=3363
Florida Legislature. (2020). Statutes & Constitution: View Statutes: Online Sunshine. Online Sunshine. http://www.leg.state.fl.us/statutes/index.cfm?App_mode=Display_Statute&Search_String=&URL=0400-0499/0408/Sections/0408.918.html
Holden, D. J., & Zimmerman, M. A. (2008). A practical guide to program evaluation planning: Theory and case examples. SAGE Publications, Inc.
13
References
Leider, J. P., Meit, M., McCullough, J. M., Resnick, B., Dekker, D., Alfonso, Y. N., & Bishai, D. (2020). The state of rural public health: Enduring needs in a new decade. American Journal of Public Health, 110(9), 1283–1290. https://doi.org/10.2105/AJPH.2020.305728
Royse, D., Thyer, B. A., & Padgett, D. K. (2016). Program evaluation: An introduction to an evidence-based approach (6th ed.). Cengage.
Ruhe, V., & Boudreau, J. D. (2013). The 2011 program evaluation standards: A framework for quality in medical education programme evaluations. Journal of Evaluation in Clinical Practice, 19(5), 925–932. https://doi.org/10.1111/j.1365-2753.2012.01879.x
14
