Interoperability Project Plan

mshp

FormativeEvaluation-the-sage-encyclopedia-of-educational-research-measurement-and-evaluation.pdf

Home >Nursing homework help >Interoperability Project Plan

Formative Evaluation

In: The SAGE Encyclopedia of Educational Research,

Measurement, and Evaluation

By: Theodore J. Christ & Jessie Kember

Edited by: Bruce B. Frey

Book Title: The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation

Chapter Title: "Formative Evaluation"

Pub. Date: 2018

Access Date: June 19, 2022

Publishing Company: SAGE Publications, Inc.

City: Thousand Oaks,

Print ISBN: 9781506326153

Online ISBN: 9781506326139

DOI: https://dx.doi.org/10.4135/9781506326139

Print pages: 697-699

This PDF has been generated from SAGE Research Methods. Please note that the pagination of the

online version will vary from the pagination of the print book.

https://dx.<wbr>doi.<wbr>org/10.4135/9781506326139

Evaluation is the process of examining a program, procedure, or product to estimate its function, effect,

and worth. There are two main functions of program evaluation in education. The first is to inform the

development and implementation of the program. The second is to estimate the outcomes and program

effects. Formative evaluation is the use of data before and/or during instruction or the implementation of

an intervention. These data are specifically used to improve and inform curriculum planning, instructional

design, and learning. The goal of formative evaluation is to meet the specific needs of students by identifying

those objectives that have and have not been mastered by the student and determining what needs to

be taught, individualizing educational programs for all students. Most importantly, formative evaluation is a

cyclical process that includes planning, managing, delivering, and evaluating instruction, learning, programs,

and interventions.

Formative evaluation allows for ongoing, real-time adaptations and modifications to aid in the development

of empirically developed and empirically informed instruction or intervention practices. While formative

evaluation aims to ensure that specific goals and objectives are being met, it also allows for improvements

to be made. Formative evaluation can involve the use of both quantitative and qualitative data. For example,

formative evaluation can rely on student performance scores on assessments or tests, and it can also rely

on students’ perceptions about an intervention that has been implemented. This entry further expands on

the definition of formative evaluation before detailing its use in education. Methods of formative evaluation

are then reviewed, followed by an examination of effective strategies and themes, and advantages and

disadvantages of formative evaluation. The entry concludes with an example of instructional design.

Defining Formative Evaluation

There has been a clear and fundamental distinction between formative and summative evaluation since

the 1960s. Summative evaluation specifically refers to evaluation completed at the end or summation of

instruction, intervention, or program activities. In contrast, formative evaluation is intended to develop and

improve a process, activity, or product in an ongoing manner, while the process, activity, or product is

active. Formative evaluation and summative evaluation differ with regard to the goals and intended use of

information. For example, summative evaluation provides information about the degree to which terminal

outcomes have been successfully attained over the course of a class, activity, program, or intervention. In

contrast, formative evaluation provides information about needs and progress during the time a program

is implemented. The underlying purpose and expected uses of the information differ. Formative evaluation

answers whether it is working; summative evaluation answers whether it worked. Finally, formative evaluation

is not the same as formative assessment. Although all formative assessment is formative evaluation, not all

formative evaluation is formative assessment. Assessment refers to the process of measuring information

about a student or program to yield a source of information. Evaluation is the process of using the information

that has been collected to make informed decisions. Put simply, assessment is the collection of information,

while evaluation is the use of that information.

SAGE

SAGE Research Methods

Page 2 of 6 Formative Evaluation

Uses of Formative Evaluation in Education

There are many potential uses of formative evaluation in an educational setting. For example, formative

evaluation can serve as a needs assessment, examining whether a program or intervention is addressing

a specific goal or objective. Formative evaluation specifically involves the use of data to identify individual

student needs. These data are then used to plan, inform, and improve academic instruction. With increased

attention toward accountability of improving outcomes for all students, the need for linking assessment to

intervention practices is irrefutable. Formative evaluation can also be used to modify instruction, a program,

or an intervention. Applying appropriate modifications during these processes allows instructors to increase

the likelihood of success. Even further, formative evaluation may be used to determine the extent to which an

intervention or program is implemented with fidelity and whether it has been implemented with consistency

and quality. Thus, in some instances, formative evaluation can serve as a means for quality control. Finally,

formative evaluation may be used to document progress on an ongoing basis in a standardized fashion,

complementing summative evaluation methods. Although formative evaluation and summative evaluation can

be successful independently, each is supplementary to the other and is more successful when used alongside

the other. A comprehensive evaluation likely includes both summative and formative practices. When used

in conjunction, these evaluation components can examine how an intervention or program was implemented,

factors that both constrained and facilitated success and effectiveness.

Methods of Formative Evaluation

Formative evaluation may include either qualitative or quantitative data. Because formative evaluation may

refer to the evaluation of activities, programs, curricula, or interventions, methods of formative evaluation are

wide and varied and may include various assessment techniques (e.g., midsemester evaluations, curriculum-

based measurement in reading), self-evaluation or self-assessments, surveys (i.e., open and close-ended

questions), focus groups or expert review, or observation techniques. Methods employed will largely be

determined by the purpose of the evaluation and the questions of interest.

Effective Strategies and Themes

For formative evaluation to be effective, there are several recommended strategies and underlying themes.

The first of these strategies is defining a specific purpose of the evaluation. The purpose should be relevant.

Even further, decisions about how the data will be used should be specified before the data are collected.

In the realm of school psychology, there are four general types of decisions that can be made regarding

individual student performance: screening, progress monitoring, analytic, and outcome. Formative evaluation

is used primarily for progress-monitoring decisions. Progress-monitoring decisions refer to those decisions

made to determine whether or not a student’s rate of progress is adequate. While this particular application

is narrow, it can be generalized to the evaluation of activities, programs, curricula, and interventions.

Regardless, there needs to be a clear and explicitly defined purpose for the program, intervention, or

SAGE

SAGE Research Methods

Page 3 of 6 Formative Evaluation

curriculum.

The second of these strategies is visual analysis. By evaluating the implementation of an intervention, data

can be visually analyzed at multiple time points throughout the intervention to determine the effectiveness.

There are four visual analysis criteria: change in mean, change in level, change in trend, and latency

of change. Change in mean refers to the extent to which the average rate of performance during the

intervention differs from the average rate of performance before the intervention. Change in level refers to

the extent to which there is discontinuity of performance when comparing baseline data to data collected

during the intervention. Change in trend refers to whether performance is increasing or decreasing throughout

the intervention. Finally, latency of change refers to the amount of time that occurs before a change in

performance is observed after implementing the intervention.

Visual analysis of data can be used to formatively evaluate student progress and to determine whether the

predefined goals will be met. For example, if the student’s performance trend is flatter than the goal line,

a decision might be made to change the implemented intervention. If the student’s performance trend is

equivalent to the goal line, a decision might be made to continue the intervention. Finally, if the student’s

performance trend is greater than the goal line, a decision might be made to increase the goal.

Third, effective formative evaluation rests on our ability to test hypotheses about instruction, learning,

programs, or interventions. This iterative process involves examining student performance frequently,

routinely, and in an ongoing manner. Evaluation is essentially founded on accurate inferences or logical

conclusions derived from a given body of evidence.

Finally, formative evaluation is a dynamic process. Generally, the more frequently educators collect data,

the better. While summative evaluation provides a static determination, decision, or diagnosis, formative

evaluation provides a responsive, data-based problem-solving strategy. Information is specifically collected

for the purpose of making decisions about instruction, learning, programs, or interventions. Formative

evaluation allows for individualized educational programs based on student performance. This inductive

and systematic approach to developing instruction and intervention allows educators to adjust the intensity,

frequency, and content. Formative evaluation relies on follow through. A process or practice is formative to

the extent that evidence about student performance is elicited, documented, interpreted, and used whether it

is used by the teacher, peers, or the learner.

Advantages and Disadvantages

Formative evaluation encourages ongoing, data-based decision making for the purpose of improving

practices, processes, plans, and programs. The information obtained throughout this dynamic process

increases the likelihood of success and allows for efficient resource allocation. In addition, formative

evaluation provides educators with a strategy for refining practices and programs that takes a preventive

approach because it takes place during the formation stage.

SAGE

SAGE Research Methods

Page 4 of 6 Formative Evaluation

However, formative evaluation does come at a cost. Formative evaluation requires time and resources.

Also, although formative evaluation increases the likelihood of intervention or program success, it also has

the ability to distort our impressions of intervention or program effectiveness. More specifically, formative

evaluation itself may be considered an intervention. Thus, it can be difficult to evaluate the independent

impact of instruction, learning, a program, or an intervention. Finally, in some instances, formative evaluation

may require making decisions and modifications with seemingly little evidence.

An Instructional Design Example

In regard to instruction, formative evaluation requires planning, managing, delivering, and evaluating. In

planning instruction, one needs to assess the baseline skill level of students before instruction occurs or

after preliminary instruction. Screening to collect student performance data can provide educators with an

assessment of the instructional environment. Next, managing instruction involves adjusting the instructional

level for individual students based on one’s assessment of the classroom environment. This includes

identifying concepts and skills that need to be taught to certain groups of students. Effectively delivering

instruction relies on continuous assessment of student mastery of the material and immediate and explicit

feedback. Finally, in evaluating instruction, instructors assess student learning and set goals for future

instruction.

See also Curriculum-Based Assessment; Curriculum-Based Measurement; Evaluation, History of;

Summative Assessment

Theodore J. Christ & Jessie Kember

http://dx.doi.org/10.4135/9781506326139.n272

10.4135/9781506326139.n272