Lac
Chapter 7: Evaluation
What is an evaluation?
An evaluation is a form of information exchange and communication among program implementers, program evaluators, funders, policymakers, and the public.
Chapter focus
Program evaluation yields results that are useful for all parties
Results from evaluations help to monitor what takes place on daily basis as well as the success of the program
Evaluations provide information about what could or should be changed, how it should be changed and why it should be changed
Information is essential for all community stakeholders including policymakers and researchers
Reasons for evaluation of afterschool programs
Accountability
When programs receive funding, they are expected to have an unbiased evaluator conduct research
Program administrators, evaluators, and funders are responsible to each other and the public to explain the various dollars spent in terms of program creation and implementation
Evaluation is one-way programs are able to fulfill this responsibility.
Sharing Results
Inform other programs about their results so that errors will not be duplicated
Programs should not compete against one another
They should communicate with one another to show what works and what does not work
The more they are able to share positive results, the more likely AS will be able to obtain more funding and become more effective
Reasons for evaluation of afterschool programs
Educational Reform
Recently school districts, foundations, and local government agencies have begun to explore the concept of AS as a part of the entire school reform movement
New school reform movement constantly investigates additional ways to improve educational opportunities and achievement levels of school-age students
The more effective evaluations completed regarding school-based extended AS programs ultimately increases funding to improve academic achievement among students by providing them with academic tutoring and homework help sessions during non-school hours
Key Components for Effective Evaluation
Designated Key Person to Collect Data (correctly)
Clear & Consistent Communication among evaluators, program implementers, and stakeholders
Working with Evaluators
a.) Inclusion
b.) Communication
c.) Potential Problem Areas (discuss before evaluation)
d.) Documentation
Four specific factors mentioned above are likely to influence the relationships between the evaluator and stakeholders
Types of Evaluation
Formative & Summative
Qualitative & Quantitative
Ongoing & Final Evaluation
Evaluations
Formative
Descriptive
Goal: gathering data that inform the readers about the components of the program (i.e. to use program must be able to track the participation of the participants on daily basis
Type of evaluation specifically explains goals
Formative evaluation is typically conducted during the development or improvement of a program or course.
Summative
Quantitative in nature
Utilizes statistical analyses
Goal: gathering data and examining data that will eventually assess program impact and effectiveness
Summative evaluation involves making judgments about the efficacy of a program or course at its conclusion.
Primary focus is outcome of program
Formative vs Summative
Evaluations
Ongoing & Final Evaluation
All evaluations should be ongoing
Results of the ongoing evaluations eventually feed the final evaluation
The stronger the ongoing evaluation, the better the final evaluation will be for researcher(s)
Evaluation design
Make sure to report all results including comparison group
(if applicable)
State limitations
Present the results with caution
Best type of evaluation
Myth: One type of research and evaluation is always better than another is!
True or False
All evaluation types work together to answer different types of questions about the evaluation
The key is for programs intending to undergo an evaluation is to be involved in all types of evaluations
Chapter take home message
Evaluations can take many different forms
Evidence of effectiveness is obtained by initially creating a robust experimental design that will provide results that are clearly attributable to the effects of the treatment and are generalizable to a specific population
Evaluating afterschool programs in an effective manner provides information that helps educate policymakers, researchers, educators, educational consumers, and all the stakeholders involved in afterschool programing about the importance and necessity of evaluation, especially if we plan to allocate large amounts of funding to said programs