9 DISCUSSIONS DUE IN 48 HOURS

combs

10CH_Teaching_Lefrancois.pdf

Home >Education homework help >9 DISCUSSIONS DUE IN 48 HOURS

309309

10Assessing Progress and Achievement

Wavebreakmedia Ltd/Wavebreak Media/Thinkstock

Learning Objectives

After studying this chapter you will be able to:

ሁ Describe the difference between measurement and evaluation. ሁ Describe the most useful kinds of instructional objectives. ሁ Name the characteristics of good measuring instruments. ሁ Analyze the advantages and disadvantages of teacher-made tests. ሁ Explain how criterion-referenced and norm-referenced evaluations are different. ሁ Define performance-based assessments. ሁ Describe how performance is reported to students and parents. ሁ Evaluate the controversy surrounding high-stakes testing.

The only man I know who behaves sensibly is my tailor; he takes my measurements anew each time he sees me. The rest go on with their old measurements and expect me to fit them.

—George Bernard Shaw

Lef83909_10_ch10_309-342.indd 309 12/14/17 4:24 PM

310

Section 10.1 Measurement and Evaluation in Schools

Pretest

Determine if the following statements are true or false.

1. Assessment includes both measurement and evaluation processes. (T/F) 2. Including more difficult items on a test improves the reliability measure. (T/F) 3. Scoring the same item on all students’ tests before scoring the next item helps make the

scoring more objective. (T/F) 4. Criterion-referenced tests evaluate students using average student performance as a

guide. (T/F) 5. Performance assessments tend to be better at assessing complex cognitive skills than

are traditional assessments. (T/F) 6. High-stakes testing can result in narrowing the curriculum that students learn. (T/F)

Answers can be found at the end of the chapter.

Introduction In spite of the intuitive appeal of “schools without failures,” “schools without tests,” and other hypothetical situations where everyone is highly motivated, absolutely dedicated, and deliriously happy, the nitty-gritty of classroom practice sometimes requires assessment. This chapter describes the various methods by which students’ and teachers’ performances can be measured and evaluated, why assessment is important, some of the abuses and misuses of assessment procedures, and some current trends in educational assessment.

When he reached twelfth grade, they sent my Uncle Robert away to a boarding school which, at that time, was home to an assortment of problem students sent there to be straightened out. Also, the school included a number of lads who were destined for the priesthood. Uncle Robert qualified on both counts.

A couple of months after school started, they sent him back home. “You said you’d do good on the tests,” said his father. “They asked stupid questions,” Robert answered. “I didn’t do so good. I just got a 43.”

10.1 Measurement and Evaluation in Schools A statement such as “I didn’t do so good” implies evaluation. To evaluate is to judge certain qualities—to place a value on them, like “good” or “bad.” A statement such as “I just got a 43” illustrates measurement. Measurement involves the use of an instrument (for example, a tape measure or a test) to assess a specific quantity. In general, measuring is a more pre- cise and more objective procedure; evaluating is less precise and more subjective. The term assessment is often used as a general term for the process of appraising student perfor- mance and can include elements of both measurement and evaluation.

Lef83909_10_ch10_309-342.indd 310 12/14/17 4:24 PM

311

Section 10.1 Measurement and Evaluation in Schools

Measurement and evaluation are important parts of the instructional process. As we saw in Chapter 1, instruction can be described as a sequence of procedures conducted before, dur- ing, and after teaching. In the before-teaching phase, assessment may be involved in placing students, selecting instructional procedures, and determining students’ readiness. During the teaching phase, assessment might be used to determine whether goals are being met, as a basis for modifying instructional procedures, or as a learning tool. And in the after-teaching phase, assessment is used not only to determine the extent to which instructional goals have been met but also to gauge the effectiveness of instructional strategies and to re-evaluate stu- dents’ placement and readiness.

Not surprisingly, assessment procedures occupy a significant portion of classroom time. Besides time spent actually writing, going over, and correcting tests, a tremendous range of other evaluative activities fills school days: asking students questions; commenting on students’ responses and presentations; appraising performances in art, drama, music, and writing; conducting informal observations of ongoing student work; correcting and grading homework assignments; and so on. In fact, most of the teacher’s evaluations, especially in the lower elementary grades, are based on informal observation and on grading assignments and performance rather than on test results.

Evaluative procedures have a profound effect on what students study and how they learn. In the final analysis, students try to learn what teachers test, not what teachers suggest is important. If teacher-made tests and final examinations ask students only to repeat what they have read in textbooks or heard in class, that is what students will learn.

Formative and Summative Evaluation The main purposes of educational assessment have traditionally been to gauge student prog- ress, to measure learning, to evaluate the learner’s qualifications, and to guide decisions about student placement. These purposes define summative evaluation. Summative evalu- ation is intended mainly to provide a grade. In a sense, it summarizes the effects of learning and instruction.

Evaluation can also be used to provide both learners and instructors with guidance designed to improve the teaching-learning process. This purpose of educational assessment defines formative evaluation. Whereas summative evaluation typically occurs at the end of an instructional unit or a school term, formative evaluation occurs earlier, sometimes beginning before instruction. And it typically continues as an integral part of the instructional process. (See Figure 10.1.)

The case “Face the Reporters” illustrates how a classroom exercise can be used as formative evaluation. “Expert” responses reveal how well each student has learned and understands the material, as do the questions the “reporters” ask. The notes Mr. Taylor takes during this class will later be useful in determining whether the entire class—or specific individuals within the class—requires further instruction, whether instructional procedures need to be modi- fied, whether content needs to be changed, and so on. These are the purposes of formative evaluation.

Lef83909_10_ch10_309-342.indd 311 12/14/17 4:24 PM

312

Section 10.1 Measurement and Evaluation in Schools

Figure 10.1: Formative and summative evaluation ሁ Flow chart showing the use of formative assessment in instruction. Note that summative

evaluation occurs at the end of the instructional sequence. Formative evaluation occurs during, and sometimes before, instruction.

Continue with planned instruction and ongoing

assessment

Use focused assessment to diagnose learning difficulties; continue remediation; modify teaching-learning

activities

Reinforce learning; offer encouragement; provide feedback from ongoing

assessment

Provide instructional intervention; implement individual remediation

or additional group activities

Ongoing formative assessment: Is

learning progressing as planned?

Summative evaluation end of instruction

Yes No

Formative assessment (monitors learner progress: Is learner achieving instructional

objectives?)

Lef83909_10_ch10_309-342.indd 312 12/14/17 4:24 PM

313

Section 10.1 Measurement and Evaluation in Schools

Instructional Goals and Assessment Educational goals tell the teacher what to teach and also which behaviors need to be evalu- ated. We cannot assess the effectiveness of our instructional procedures unless we know what they are intended to accomplish, and we cannot determine what they have accomplished without some form of assessment.

The desired outcomes of education can be expressed in broad terms such as “the goal of edu- cation is to develop decent, worthwhile citizens” or “the goal of education is to empower students.” Table 10.1 presents the broad educational goals outlined by the U.S. Department of Education (About Ed: The Department’s FY 2016–2017 Priority Performance Goals, 2015). But although general statements of objectives such as those listed in Table 10.1 are useful for guiding the development of curriculum and influencing the behavior of administrators and students, they are not nearly as useful in the day-to-day business of teaching as are more specific instructional objectives.

C A S E S F R O M T H E C L A S S R O O M : F A C E T H E R E P O R T E R S The Place: Mr. Taylor’s 12th-grade biology class

The Situation: A lesson on Mendelian genetics

Mr. Taylor: Experts, take your places!

(Four pre-selected students take the chairs that have been set in front of the class. Their assignment was to become experts on Mendelian genetics. Today, when they face the reporters—the remaining class members—they will be expected to answer questions as though they were knowledgeable biologists. James has been selected as moderator.)

James: Okay. Our first reporter is Brian.

Brian: Mr. Robert. Could you tell us briefly who Gregor Mendel was and what he did?

Robert: He was a close friend of mine back in . . . at the turn of the 20th century. He was a professor in Russia and he studied fruit flies.

James: Is that correct, Ms. Williams?

Sandra Williams (another expert): Well, not exactly. He was born before the 19th century. And he was Austrian, and it wasn’t him that studied fruit flies. He cultivated peas and that’s what led him to the laws of heredity.

Sally (a reporter): Can two people with brown eyes have a daughter with blue eyes, and why, or why not?

Experts now take turns talking, using terms like recessive, heterozygous, homozygous, genotype, and phenotype. William projects a “Punnett square” on the smart board to show all possible combinations. Reporters ask a series of questions: experts answer as they can. And Mr. Taylor takes notes and also clarifies and amplifies as needed or asks other stu- dents to do so.

Lef83909_10_ch10_309-342.indd 313 12/14/17 4:24 PM

314

Section 10.1 Measurement and Evaluation in Schools

Instructional objectives are statements about the type of performance that can be expected of students once they have completed a lesson, a unit, or a course. Objectives do not describe the course itself but describe instead the intended performance of students. Because perfor- mance implies behavior, the phrase “behavioral objectives” is sometimes used interchange- ably with “instructional objectives.”

Mager’s Instructional Objectives To be useful, says Mager (1997), instructional objectives must specify clearly what the learner should be able to do following instruction. That is, the instructor’s goals should be worded in terms of the actual, observable performance of the student. These behavioral objectives serve as a description of course goals and as a guide for instructional strategies. They describe specific, observable behaviors as goals of the instructional process, and they also serve as a guide for assessing the performance of students and teachers.

Consider the following statements of instructional objectives:

1. The student should understand evolutionary theory. 2. The student should be able to state the two Darwinian laws of evolution and give

examples of each.

Mager argues that the second objective is more useful than the first because it specifies exactly what students must do to demonstrate that they have reached the course goal. Furthermore, it provides the teacher with specific guidelines for determining whether course goals have been reached, and it suggests what must be taught if course goals are to be reached.

Table 10.1: Educational goals listed by the U.S. Department of Education

General goal Specific related goal

Increase college degree attainment in America By 2017, 48.4% of adults aged 25–34 will have an associate degree or higher

Federal student aid transparency The publication of information about student loan portfolios for taxpayers, researchers, and the public

Implementation of college- and career-ready standards and assessments

Ensure that all states use high-quality assessments aligned with college- and career-ready standards

Increase enrollment in high-quality state preschool programs

Increase percentage of children (especially from low- income families) in preschool programs, and increase the number of high-quality programs available

Ensure equitable educational opportunities Decrease the number of schools with low graduation rates; develop strategies to ensure high-quality teachers in all schools

Enable evidence-based decision making Increase grants to support evidence-based approaches to education

Source: About Ed: The Department’s FY 2016–2017 Priority Performance Goals (2015). U.S. Department of Education. Retrieved October 25, 2016, from http://www2.ed.gov/about/overview/focus/goals.html

Lef83909_10_ch10_309-342.indd 314 12/14/17 4:24 PM

http://www2.ed.gov/about/overview/focus/goals.html

315

Section 10.1 Measurement and Evaluation in Schools

A second characteristic of meaningful instructional objectives is that they establish specific criteria for meeting goals. Consider the following statements:

1. The learner will be able to translate a simple passage from Spanish to English. 2. The learner will be able to translate a simple passage from Spanish to English with-

out use of a dictionary. The passage will be taken from the prescribed text, and the translation will be considered correct if there are no more than 5 errors for each 100 words of text and acceptable if the translation is completed in no more than 20 min- utes for each 100 words.

The second statement is more precise than the first, and again it is more useful for both the instructor and the learner.

Gronlund’s Objectives Not all educators agree that Mager’s approach to instructional objectives is the best. Emphasis on precise, performance-oriented objectives is appropriate for simple skills and for content areas that can be described in terms of specific items of information, says Gronlund (Gron- lund & Brookhart, 2009). But such emphasis is considerably less appropriate for more com- plex subjects and more advanced cognitive behaviors. In addition, performance objectives do not recognize the importance of affective outcomes.

Accordingly, Gronlund recommends that teachers express main objectives in general rather than specific terms. Each main objective should then be elaborated in terms of more specific learning outcomes or, in many cases, in terms of actual examples of behaviors that would reflect the primary objective. These examples can then be used as a basis for what is termed performance assessment or authentic assessment (Gronlund & Waugh, 2013). (See Figure 10.2.)

Figure 10.2: Two approaches to writing instructional objectives ሁ Gronlund advocates deriving specific instructional objectives from more general objectives; Mager

suggests that the most useful objectives are those that specify precise learner behaviors that can be observed and assessed.

Primary objective: The student understands how to

perform simple multiplication.

Specific learning outcomes:

a. can define what multiplication means in his or her own words

b. can define relevant terms such as multiplier and product

c. can solve multiplication problems involving any combination of

single digits 1 through 9

Gronlund: “General and specific”

The student can solve nine out of ten multiplication problems of the type:

5 X 4 = ?

Mager: “Behavioral objective”

Lef83909_10_ch10_309-342.indd 315 12/14/17 4:24 PM

316

Section 10.2 Characteristics of a Good Measuring Instrument

Blueprints for Test Construction Rulers, thermometers, and bathroom scales measure whatever they measure directly; our psychological and educational instruments measure indirectly. What they assess are samples of behaviors selected from many potential behaviors that are assumed to represent things like knowledge, ability, skills, or other characteristics. The question of precisely which behav- iors to sample is extremely important.

Teachers generally have a pretty good idea of the sorts of things they want their students to learn. That is, they know (or should know) what their instructional goals are. In preparing tests, they try to select a sample of behaviors that will reflect attainment of these instructional goals. For this purpose, it’s very useful to systematically blueprint tests even before teaching the relevant series of lessons.

A test blueprint is a specific plan for a test. It is based on course objectives and details the topics to be tested, the nature of the questions to be used, how many questions will relate to each topic, and the sorts of cognitive processes to be sampled (Popham, 2017). Test blue- prints need not be developed only by teachers; they can also involve the collaboration of stu- dents. Constructing the blueprint can do a great deal to clarify instructional goals, both for the teacher and for students. It can also contribute in important ways to the teacher’s selection of instructional strategies and to the students’ monitoring of their own learning processes.

10.2 Characteristics of a Good Measuring Instrument Probably the most important characteristic of a good test from the students’ point of view is that it be fair. In part, this means that the test should reflect instructional objectives as they are understood by students—that is, it should reflect what was to be learned (and, presum- ably, what was actually taught). Also, as Reynolds, Livingston, and Willson (2009) note, lan- guage difficulties such as those faced by minority-language students sometimes make tests seem grossly unfair. Furthermore, tests are unfair when they assess knowledge or skills that not all have had an equal opportunity to learn.

From a measurement point of view, good measuring instruments have two other important qualities: validity and reliability.

Validity A test is valid if it measures what it is intended to measure; many tests don’t, or else they measure many other things as well and are consequently not very dependable. From a mea- surement point of view, validity is the most important characteristic of a measuring instru- ment. If a test does not measure what it purports to, the scores derived from it are of no value whatsoever.

Face Validity There are several different ways to measure or estimate validity, three of which are important for teacher-made tests (see Figure 10.3). The first, face validity, is the extent to which the test appears to measure what it is supposed to measure. Students should know just by looking at a test that they are being tested on the appropriate things.

Lef83909_10_ch10_309-342.indd 316 12/14/17 4:24 PM

317

Section 10.2 Characteristics of a Good Measuring Instrument

In some circumstances, however, test makers carefully avoid any semblance of face valid- ity. Tests designed to measure personality characteristics, such as honesty or narcissism, for example, are likely to be highly invalid if they appear to measure what they are actually intended to measure. Because we know that dishonest people might well lie to us, we are not likely to obtain an accurate measure of their honesty if we let them know what we are mea- suring. Better to deceive them to determine what liars and scoundrels they really are.

Content Validity A test should not only look like it measures what it says it measures (face validity), but it actually should measure the right things: it should have content validity. A test with high content validity includes items that sample all important course objectives in proportion to their importance.

Predictive Validity One of the important uses of tests is to predict future performance. Thus, we might assume that all students who do well on year-end fifth-grade achievement tests will do reasonably well in sixth grade. The extent to which our predictions are accurate reflects predictive valid- ity. It is easily measured by looking at the relationship between test performance and sub- sequent performance. A college entrance examination designed to identify students whose chances of success in college are good has predictive validity to the extent that its predictions are borne out.

Reliability Good measuring instruments are not only valid, they are also reliable. A reliable test is one that measures consistently. An intelligence test that yields a score of 170 for a student one week and a score of 80 the next week is probably somewhat unreliable (unless something quite dramatic happened to the student during the week).

Figure 10.3: Types of test validity ሁ Validity is the extent to which a test measures what it is intended to measure. Its validity may be

evident in its appearance (face validity), the nature of its items (content validity), and how well it foresees future performance (predictive validity).

Test validity

Face validity

The test appears to measure what it says

it measures.

Content validity

The test samples behaviors that represent both the topics and the

processes implicit in course objectives.

Predictive validity

Test scores are valuable predictors of future

performance in related areas.

Lef83909_10_ch10_309-342.indd 317 12/14/17 4:24 PM

318

Section 10.2 Characteristics of a Good Measuring Instrument

Types of Reliability A test that is highly unreliable cannot be valid. Put another way, if a test measures what it purports to, and if what it measures does not fluctuate unpredictably, the test should yield similar scores if given more than once. Hence, one way to assess reliability is to look at the correlation between results obtained by giving the test twice or by giving two different forms of the same test to the same individuals. These are called repeated-measures reliability and parallel-forms reliability.

Another way to determine reliability, called split-half reliability, is to administer a single test and then divide it into two halves for scoring (say odd-numbered items in one half and even-numbered items in the other). If all items are intended to measure the same things, the scores on the two halves should be similar.

Factors That Affect Reliability One factor that contributes to the reliability of a test is the stability of what is being measured. Clearly, if a characteristic fluctuates dramatically over time, measurements of that character- istic will also fluctuate. However, most of what we measure in psychology and education is not expected to fluctuate unpredictably.

A second important factor, particularly with respect to teacher-constructed, multiple-choice, or true–false tests, is chance. In a true–false test, for example, the chance of getting any single item correct, if the student knows absolutely nothing, is one out of two. Therefore, unless Lady Luck is looking pointedly in the other direction, the average score of a large class of hypothetical know-nothings should be about 50 percent. And some of the luckier individu- als in this class may have astonishingly high scores. But a subsequent administration of this examination might lead to startlingly different results.

One way to increase test reliability is simply to make tests longer. Longer tests are gener- ally more reliable because, ideally, they sample a wider range of relevant content and skills than do shorter tests. The important point is that the test needs to include sufficient items to adequately sample behaviors reflecting instructional objectives. But, of course, increased reliability also depends on the quality of the items, their difficulty, and the range of resulting scores.

Another way of increasing reliability is to use a preponderance of items of moderate difficulty, rather than very difficult items or very easy items (Lefrançois, 2013). Using items that are very difficult or very easy limits the range of scores that will be obtained, restricts the variety and extent of student learning that can be assessed, and tends to result in less consistent pat- terns of responding—hence, lower reliability. In addition, using overly difficult items leads to guessing, which increases the effects of chance and lowers reliability.

Standardized Tests Standardized tests are professionally prepared tests that provide standards (also called “norms”) by which to judge the performance of individual students. Intelligence tests, for example, are standardized so that average performance is reflected in a score close to 100 and so that the distribution of these scores in an average population is known.

Lef83909_10_ch10_309-342.indd 318 12/14/17 4:24 PM

319

Section 10.3 Teacher-Made Tests

Among standardized tests that are particularly important for classroom teachers are stan- dardized achievement tests. These professionally developed tests are available for virtually every school subject and grade level and are designed to provide information about the rela- tive performance of individual students, classes, or schools. Standardized achievement tests are mandatory for different subjects and grade levels in many school districts. They typically provide one of several kinds of norms including grade-equivalent scores, age-equivalent scores, or percentiles.

When raw scores on a standardized reading test are converted to grade-equivalent scores, teachers and parents have some basis for concluding that a student is reading at a grade level of 3, or 3.5, or 5, or whatever. Grade-equivalent scores are common norms for many standard- ized achievement tests.

Age-equivalent scores can be interpreted to mean that a student is functioning at a level com- parable to the average for a specific age group. Age-equivalent scores are more common for intelligence tests and other measures of ability or aptitude than they are for achievement tests. This is true largely because it is more meaningful to say that a person is intellectually at the level of a 4-year-old or a 9-year-old than to say that someone reads at a 4-year-old or a 9-year-old level.

Percentiles are a third common type of norm for standardized tests. A percentile is the value at or below which a certain percentage of cases fall. For example, on a standardized achieve- ment test, the 90th percentile is the score at or below which 90 percent of all scores fall.

The main purpose of standardized tests has always been to compare the performance of a single test taker to the average or expected performance of a comparable group—namely, the group on which the test was standardized. This permits schools and teachers to use tests for several different purposes.

One purpose is for placement in special education programs. Intelligence tests, which are a common form of standardized ability test, are widely used for this purpose.

A second use of standardized tests is to certify students’ achievement. Minimum competency tests are a clear example of this use, as are standardized entrance examinations used to decide whether to accept an applicant for a course of study, and the standardized final examinations used to determine success or failure following a course.

Standardized tests are now widely used to select students for teacher training and for certi- fication (the Praxis® tests, for example). They are also used to evaluate schools (through the performance of students), and for instructional diagnosis. These are instances of high-stakes testing (discussed in more detail in a later section).

10.3 Teacher-Made Tests Some of the tests used in the classroom are constructed by classroom teachers; many, how- ever, are tests—or items—prepared by others (Frey & Schmitt, 2010). Some teacher-made tests are highly representative of course objectives, are at an appropriate level of difficulty, and are used reasonably and wisely; others, not so much.

Lef83909_10_ch10_309-342.indd 319 12/14/17 4:24 PM

320

Section 10.3 Teacher-Made Tests

Teacher-made tests are often used for summative evaluation—that is, to assign grades. But perhaps far more important is formative evaluation where tests are used to determine whether students are ready to begin a unit of instruction, to indicate to the teacher how effec- tive instructional procedures are, to identify learning difficulties, to determine what students know and what they don’t know, to motivate students to learn—and as a learning experience. In short, formative evaluation is the use of assessment to improve teaching and learning.

Teacher-made tests may be of the paper-and-pencil or computer-based variety, although sometimes a sample of nonverbal behavior might be used for assessment. For example, in physical education, art, drama, and some workshop courses, students are often asked either to produce something or to perform. Increasingly, performance-based assessment may also be required in other subjects such as mathematics, where teachers are interested in examin- ing the processes by which students arrive at their answers rather than simply the products (more about performance assessment later in this chapter).

Written tests may be objective tests or essay tests. An essay test requires a written response of some length for each question. Objective tests, however, normally require little writing, and the scoring procedure is highly uniform (hence, objective).

The four major types of objective test items are matching, true–false, multiple choice, and supply type, which includes completion and short answer. Examples of each are given in Fig- ure 10.4.

Essay Versus Objective Tests There are several important differences between essay and objective tests. These differences can serve as a guide when deciding which to use in a given situation. Often, a mixture of both can be used to advantage.

For example, it is easier to tap high-level processes (analysis, synthesis, and evaluation) with an essay examination. In fact, some processes, like writing a poem or learning keyboard- ing skills, cannot be measured with multiple-choice items. Also, essay exams allow for more divergence. Students who do not like to be restricted in their answers often prefer essays over more objective tests. However, Bleske-Rechek, Zeug, and Webb (2007) found that very few students consistently do better on one type of test than on another.

The content of essay examinations is often more limited than the content of the more objec- tive tests. Multiple-choice items, for example, allow teachers to sample a very broad range of content. Because essay exams usually consist of fewer items, the range of abilities or of infor- mation sampled may be reduced.

Constructing an essay test is considerably easier and less time-consuming than making up an objective examination. However, scoring essay tests is far more time-consuming than scoring objective tests, especially if classes are large. The relationship between total time involved in making and scoring a test for different size classes is shown in Figure 10.5.

Lef83909_10_ch10_309-342.indd 320 12/14/17 4:24 PM

321

Section 10.3 Teacher-Made Tests

Figure 10.4: Types of objective tests ሁ The four major types of objective test items with examples of each. Objective tests are easy to

score quickly and accurately but are time-consuming to develop.

3. True–False

1. Supply Type

4. Multiple Choice

2. Matching

Completion: a.) __________ evaluation is intended to provide a grade. b.) Evaluation intended to guide the teaching-learning process is _________________.

Short answer: a.) What are professionally constructed and normed tests called? b.) A measure of the extent to which a test tests what it is intended to test is called:

_______valid _______reliable _______fair

1. neither rewards nor punishes unduly 2. measures what it purports to 3. measures accurately

a.) A good achievement test should result in a grade-equivalent score between 4 and 5 for an average fourth-grade class.

b.) Content validity can be determined by making a careful, logical analysis of the relationship of test items to course objectives.

The extent to which a test appears to measure what it is intended to measure defines:

a.) content validity b.) face validity c.) predictive validity d.) criterion-related validity

Lef83909_10_ch10_309-342.indd 321 12/14/17 4:24 PM

322

Section 10.3 Teacher-Made Tests

Note that the reliability of essay examinations is much lower than that of objective tests, pri- marily because of the subjectivity involved in the scoring.

Suggestions for Constructing Tests The advantages of a particular type of test can often be increased if the test is constructed carefully. By the same token, the disadvantages can also be made more severe through faulty item construction. Essay examinations, for example, can be better for measuring “higher” mental processes. But consider the following item:

List the kinds of validity discussed in this chapter.

If the tester’s intention is to sample analysis, synthesis, or evaluation, this item has no advantage over many objective items. However, an item such as the following might have an advantage:

Discuss similarities and differences among the different types of validity discussed in this chapter.

Following are several specific suggestions for constructing essay tests and multiple-choice tests, the most preferred among objective-item formats.

Figure 10.5: The relationship between class size and time required for test construction and scoring

ሁ With larger classes, scoring time increases dramatically for essay-type tests but does not change appreciably for objective tests. Initial test preparation time is typically much lower for essay tests than for objective tests.

To ta

l t im

e fo

r c on

st ru

ct io

n an

d sc

or in

g fr

om lo

w to

h ig

Number of students from low to high

Essay

Objective

Lef83909_10_ch10_309-342.indd 322 12/14/17 4:24 PM

323

Section 10.3 Teacher-Made Tests

Essay Tests The following several suggestions are based in part on Gronlund and Waugh (2013):

1. Essay questions should be geared toward sampling processes not easily assessed by objective items (for example, analysis, synthesis, or evaluation).

2. As with all tests, essay questions should relate directly to the desired outcomes of the learning procedure.

3. To be scored easily, questions should be specific. If the intention is to give marks for illustrations, the item should specify that illustrations are required.

4. A representative sampling of desired behavior should make up the substance of the items. In other words, the items selected should not all tap the same content or skill areas.

5. Sufficient time should be allowed for students to complete the questions. 6. The weighting of various questions, and the time that should be allotted to each,

should be specified for the student. 7. The questions should be worded so that the teacher’s expectations are clear to both

the student and the teacher.

There are ways to make the scoring more objective as well. One is to outline model answers before scoring the test (that is, write out an answer that would receive full points). Another is to score all answers for one item before going on to the next; this should increase uniformity of scoring. A third suggestion is simply that the scorer be consistently objective. For example, if poor grammar results in the loss of 5 points on one paper, grammar that is half as bad on another paper should result in the loss of 2½ points.

Multiple-Choice Items A multiple-choice item consists of a statement or series of statements (called the stem) and three to five alternative responses. Generally, only one of the alternatives is the correct or best solution; the other alternatives are distractors. However, Kubinger and Gottschall (2007) constructed multiple-choice items in which any number of the five alternatives might be correct and the item would be deemed to be answered correctly only if all correct alter- natives were chosen. They found that these items were more difficult than identical items where only one alternative was correct because they had effectively eliminated the benefi- cial effects of guessing.

Listed here are several suggestions for writing multiple-choice items. Most involve common sense (which makes them no less valid):

1. Each of the distractors should appear plausible if students do not know the answer. If students do know the correct answer, distractors should, of course, appear less plausible.

2. Both stems and alternatives should be clearly worded, unambiguous, grammatically correct, specific, and at the appropriate level of difficulty.

3. Don’t use no double negatives if constructing tests in the language you are now read- ing. In English (though not in French or Spanish), double negatives are very confus- ing and should be avoided. Are single negatives highly recommended? Not.

Lef83909_10_ch10_309-342.indd 323 12/14/17 4:24 PM

324

Section 10.3 Teacher-Made Tests

4. Test items should sample a representative portion of subject content, but they should not be taken verbatim from the textbook. This is defensible only when the intention is clearly to test memorization.

5. Provide students with feedback shortly after testing. Not only does going over and explaining test items lead to more learning, but it can also reduce the false learning that sometimes occurs as a result of selecting incorrect answers. Providing learners with immediate feedback can do much to improve learning and reduce false learning (Butler & Roediger, 2008).

6. All distractors should be equally plausible so that answering correctly does not become simply a matter of eliminating highly implausible distractors. Highly implau- sible distractors can inflate scores dramatically.

7. Unintentional cues should be avoided. For example, ending the stem with a or an often provides a cue.

8. Qualifying words such as never, always, none, impossible, and absolutely should be avoided in distractors (though not necessarily in stems). They are almost always asso- ciated with incorrect alternatives. Words such as sometimes, frequently, and usually are most often associated with correct alternatives. In stems, both kinds of qualifiers tend to be ambiguous.

Making Test Results Meaningful The main purpose of assessment, today’s educators insist, is not so much to audit students but rather to support and assist teaching and learning. But this insistence may represent more of a shift in rhetoric than in actual practice. In fact, notes Keeley (2008), in this climate of accountability and high-stakes testing, much of educational assessment continues to be sum- mative (it summarizes student progress and achievement) rather than formative (directed specifically toward improving teaching and learning).

No matter what the purposes of assessment, results typically need to be reported to students and parents. Although actual test scores can be reported directly to the student, these raw scores might not be very meaningful. A score of 40 on a test with a maximum possible score of 40 is different from a score of 40 on a test in which the ceiling is 80.

The traditional way to give meaning to test scores is to convert them either to a percentage or to a letter grade that has clearly defined (though arbitrary) significance. And if students are also told the mean (arithmetical average) and the range of scores (low and high scores), the scores become even more meaningful.

The mean is called a measure of central tendency because it indicates the approximate center of a distribution of scores. Two other common measures of central tendency are the median and the mode. The median is the exact midpoint of a distribution. It is the 50th per- centile—the point at and below which 50 percent of all scores lie. The mode is simply the most frequently occurring score. As such, it is not particularly valuable for educational and psychological testing; while teachers may be interested in the average (mean) or even the midpoint (median) performance of learners, it is seldom important for them to know that, say, 54 or 73 was the most frequently recurring score. In contrast, the mode is of considerable interest to shoe and clothing manufacturers, because they are not interested in manufactur- ing average or median sizes, but rather those that occur most often.

Lef83909_10_ch10_309-342.indd 324 12/14/17 4:24 PM

325

Section 10.4 Criterion-Referenced Testing

10.4 Criterion-Referenced Testing Ochawa is a small kingdom hidden in a steamy jungle somewhere. One of its borders is a great river that describes a serpentine half-circle around most of its perimeter; the other border is an impenetrable row of mountains. So the inhabitants of Ochawa are trapped between the river, which they can’t cross, and the mountains, which they can climb but not cross because the other side is an unbroken row of sheer rock. In Ochawa are many extremely ferocious, people-eating beasts. Fortunately, all are nocturnal. I say “fortunately” because, although the human inhabitants of Ochawa live on the mountainsides well beyond where the predators can climb, every day they must descend the mountain to find food.

Testing in Ochawa In Ochawa a test is given to all able-bodied men, women, and children each day of their lives. It’s a simple test: Before nightfall, each must succeed in climbing the mountain to a point beyond the reach of the predators. Fail- ure to do so is obvious to all because the indi- vidual who fails simply does not answer roll call that evening. Success is equally obvious.

This test is quite different from the ordinary testing practices of most schools. Passing the test doesn’t require that individuals be the first to reach safety; it doesn’t even require that they be among the first 90 percent to do so. Nor do they need to climb higher or more gracefully than everyone else. In fact, they will be just as successful if they are among the very last to claw their way to safety. They will be just as alive as the first (and maybe less hungry).

Testing in Schools Consider the situation in most of our schools. Assume that all students are expected to attain a certain level of performance in a variety of subjects—a level of performance that we will denote by the symbol X. Reaching X is analogous to climbing the cliff before the snarling beasts grab you.

But when assessing students’ performance and reporting grades, teachers don’t often ask themselves which students have reached X and which haven’t. Instead, they compare the performance of each child to the average performance of all children and then make judg- ments about the relative performance of their students. In a very advanced class, students who have in fact reached X but who fall well below average performance may be assigned mediocre marks. In a less advanced class, the same students might be assigned much higher grades. This illustrates what is labeled norm-referenced testing. Norm-referenced tests

Liu Yin/Hemera/Thinkstock ሁ There are cliffs here that the beasts of Ochawa

cannot climb. But if you are a resident of Ochawa, you must climb them every day. However, you don’t need to climb higher, faster, or earlier than anyone else. When you reach high enough, you will be just as safe and happy as everyone else. In a sense, the climb is a criterion-referenced test.

Lef83909_10_ch10_309-342.indd 325 12/14/17 4:24 PM

326

Section 10.4 Criterion-Referenced Testing

are tests in which the student’s performance is judged and reported in terms of a standard or norm that is derived from typical student performance on the test. Such testing is highly compatible with competitive approaches to teaching but adapts poorly to more cooperative approaches. Norm-referenced testing, says Kingston (2008), is the dominant form of testing in most schools.

There is a second alternative in which students are not compared one to the other; instead, their performance is judged only in relation to a criterion—hence, criterion-referenced testing. In the jungle example, the criterion is simply the ability to climb beyond the reach of predators; success is survival, and failure is death.

Criterion-referenced testing can be used in schools and is in fact used extensively with Bloom’s mastery learning, Keller’s PSI, and other forms of individualized instruction (see Chapter 7). For this reason, criterion-referenced tests are sometimes called mastery tests (Finkelman, 2010). Criterion-referenced testing allows teachers to determine whether a particular stu- dent has reached (mastered) a predetermined level or standard without having to compare that student to any other.

What is required for criterion-referenced testing is that teachers be able to specify what is involved in achieving X. That is, they must be able to specify the criteria of success in terms of specific behaviors. Often, these behaviors are students’ responses to test items. Hence, criterion- referenced testing typically involves careful analysis of items in light of clear instructional objec- tives. One of the great benefits of this, claims Popham (2014), is increased clarity of objectives and of test items—and perhaps increased effectiveness of instruction as well.

Which Approach Is Best? The tests used for criterion-referenced and norm-referenced testing can be identical, explains Popham (2014); the difference between the two lies in how teachers use them rather than in the nature of the tests themselves. That is, criterion-referenced assessment has to do with how a test is interpreted rather than with the nature of the test itself. With criterion-referenced testing, each student’s performance is compared to a criterion—that is, to a pre-determined standard of knowledge or skill. But with norm-referenced testing, an individual’s performance is compared to that of other students. Individual differences are far less important in criterion- referenced testing. Indeed, the objective is to have all students succeed. Hence, the principal use of criterion-referenced tests is to test for mastery; norm-referenced tests are more useful for surveying the performance of a group of learners and for comparing an individual to the group.

Advocates of criterion-referenced testing point to the inherent justice of their approach: No student need consistently fail for performing less well than others after a predetermined period of time. When students reach the criterion, they will be as successful on that particular task as all other students. And students who have more to learn at the outset of instruction will not fail simply because they start at a different place and consequently lag behind others in the beginning. If they reach the mountain heights before the beasts, they will survive just as surely as will those who climbed first, fastest, and highest. Criterion-referenced testing argues strongly for the individualization of instruction and of evaluation. It fosters coopera- tion rather than competition, it encourages students to work toward the goals of the system rather than against other students, and it forces teachers to make those goals explicit.

Lef83909_10_ch10_309-342.indd 326 12/14/17 4:24 PM

327

Section 10.5 Performance-Based Assessments

But criterion-referenced testing has certain limitations, as critics have been quick to point out. First, it is not always possible. For example, while it is relatively simple to specify that after taking typing lessons for six weeks a student should be able to input 30 words per min- ute with no more than two errors, it is considerably more difficult to determine precisely what a student should know or understand after sitting in a social studies class for six weeks. A criterion-referenced test is clearly appropriate in the first instance but is more difficult— though not impossible—in the second. Among other things, teachers are required to specify their standards and criteria with a clarity that is not always easily achieved.

A second limitation is that some students are able to go beyond the criteria, given sufficient motivation to do so. Some educators fear that exclusive reliance on criterion-referenced test- ing can thwart students’ initiative and stifle their motivation.

One advantage of norm-referenced testing is that it provides both students and those who would counsel them with valuable information concerning their likelihood of success in aca- demic situations where they will, in fact, be required to compete with others.

What should you do while the controversy rages around you? Nothing dramatic—this isn’t exactly a war, so you don’t have to take sides. Simply use both types of test. As Lok, McNaught, and Young (2016) explain, these two approaches to assessment are highly compatible and should be used with each other. There are situations where norm-referenced tests are unavoidable and also highly useful. There are also many situations where justice and good sense dictate that you should establish definite criteria for success. Your students’ learning— and your teaching—will benefit from your wisdom.

10.5 Performance-Based Assessments Recent literature on educational assessment contains ever-increasing references to what are often termed new approaches to assessment. And although many of these approaches have been around for a long time, their popularity has increased dramatically in recent years. Collectively, these approaches are termed performance-based assessment (or simply perfor- mance assessment). They are sometimes referred to as authentic assessment. The grow- ing popularity of performance-based assessment is based largely on its promise that it can provide more authentic, direct appraisals of student competence and that it will therefore have clearly positive consequences for learning and teaching (VanTassel-Baska, 2014).

Essentially, performance-based assessment attempts to determine students’ competence by looking at examples of what they do—that is, by looking at their actual performance. Per- formance assessment requires that actual levels or standards of performance be expressed in terms of concrete examples and explicit definitions. The assumption is that performance assessment requires a level of understanding and an ability to apply learning that is not always evident in more conventional forms of assessment. Farr and Tone (1998) point out that a conventional, multiple-choice test might determine that carpentry students know the difference between a ball-peen, sledge, or claw hammer, but it would not demonstrate that the students actually know how to build a house. In their words, “There’s a big difference between naming hammers and pounding nails in a wall” (p. viii).

Lef83909_10_ch10_309-342.indd 327 12/14/17 4:24 PM

328

Section 10.5 Performance-Based Assessments

Why Performance-Based Testing? Besides teaching reading, writing, arithmetic, and other related things, one of the purposes of your instructional procedures may be something as vague as “to teach students to use their minds well”—or something as specific as to teach them to hit a golf ball where they want to. If that is what you really want to do, then your assessment procedures should not be limited to items of information well remembered—or forgotten—but should include actual evidence of minds working well—or of golf balls landing near the pin. Phrased another way, if one of the important purposes of education is to empower students to become autonomous, reflec- tive, thinking learners, and perhaps to become accomplished, well-rounded artists, athletes, or debaters as well, then we absolutely need assessment procedures that look at the extent to which these goals are being met.

Types of Performance-Based Testing Performance-based assessment includes a variety of procedures and approaches.

Developmental Assessments Developmental assessments attempt to document evidence of progressive accomplish- ments. They look at characteristics of a student’s actual behavior over a period of time rather than at accomplishments relative to other, comparable students.

Developmental assessments typically use detailed, performance-based lists of expected or desired behaviors as a guide for evaluating a student’s progress. The most common approach to developmental assessments takes the form of checklists that present detailed, sequential lists of accomplishments or capabilities in any of a variety of areas. In some school systems, checklists have completely replaced more traditional, grade-oriented assessments.

Less rigorous approaches to developmental assessment have long been common in elemen- tary schools. There, elementary teachers rely more on observation and relatively informal evaluation. That is, they make informal judgments about the competency of their students based on their observations of students’ actual performance. In fact, informal assessments are highly common in subjects like art, drama, and physical education. Unfortunately, how- ever, as Hay and Macdonald (2008) discovered when they interviewed and observed phys-ed teachers and their students, these assessments are often made with very little substantial evidence. They report that assessing student competence was often a matter of “intuition,” and that irrelevant characteristics of students and pre-established teacher expectations con- tributed significantly to the assessments.

Here, as in other forms of performance assessment, it’s important that standards of accept- able performance be clearly understood both by teachers and by students. One of the advan- tages of well-designed developmental checklists is that they specify the physical, motor, and/ or cognitive accomplishments that define progress.

Lef83909_10_ch10_309-342.indd 328 12/14/17 4:24 PM

329

Section 10.5 Performance-Based Assessments

Demonstrations and Exhibitions The use of demonstrations is a form of performance-based assessment that requires stu- dents to answer questions, solve problems, write, or do other things in actual rather than contrived circumstances. The objective is to sample performance that provides evidence of competency in the skills involved. Assessment through demonstration is what happens when a novice driver takes a road test or a singer auditions. In these cases, the objective is to dem- onstrate the ability to perform the activity being assessed by actually performing it. In much the same way, if we want to assess a student’s ability to think, we need to sample actual per- formances of thought. This might be the case, for example, when students are asked to solve mathematical problems by being given play money with which to “buy” objects in a “store.”

Closely related to demonstrations are exhibitions. Exhibitions are public displays of per- formance. They differ from other performance-assessment procedures in that they under- line the social nature of thinking and learning. Common examples of exhibitions include oral examinations by expert or peer committees, musical recitations, and science fairs. This form of assessment requires students to synthesize their knowledge, to extrapolate from it, and to explain and demonstrate this knowledge in social situations.

Portfolios Portfolios are collections of actual samples of a student’s work, often gathered by the stu- dents themselves. In the primary grades, for example, portfolios might consist of drawings (preferably with the student’s story/explanation written on the back, often by the teacher); samples of the student’s writing; records of simple computations; perhaps tests and quizzes that tap meaningful processes. In later grades, portfolios include not only samples of early and later work but also a more systematic and structured range of work representing vari- ous subjects and cognitive processes. They are, in a sense, ongoing records of achievement or performance.

There is evidence that portfolios can improve student performance and motivation (Burks, 2010). Their use often seems to generate a more positive response in learners than more traditional forms of assessment, leading to greater interest in learning and, therefore, higher motivation. Axton (2012), following an investigation of the role of portfolio assess- ments in a mandatory English class, suggests that increased motivation may result from the self-reflection that typically accompanies the use of portfolios. In his study, interviews with learners indicated increased pride in their work and significantly more positive atti- tudes toward the course.

The case “My Lame Excuse” illustrates how portfolios can be used not only to motivate but also to teach and to evaluate. For example, after Robert has finished reading his story, stu- dents and teacher share ideas for improvements and discuss the principles of good essay writing (instructional function). Later, they hand in their portfolios so that Ms. White can consult them in preparing upcoming report cards (evaluative function). But in the end, these belong to the students and are always returned to them. Some of these now-adult students still prize their “dossiers” very highly (motivational function).

Lef83909_10_ch10_309-342.indd 329 12/14/17 4:24 PM

330

Section 10.5 Performance-Based Assessments

Rubrics A rubric is a written set of guidelines that specifies the criteria and the standards that might be used to assess the quality of a performance or a product. As Larkin (2008) puts it, “a rubric is a measurement and instructional tool that communicates instructor expectations to learn- ers” (p. 859). The advantage of a rubric is that it allows for more consistent and fairer assess- ment. Also, because rubrics are typically given to learners beforehand, they serve as guides in directing learner efforts. In fact, a study reported by Andrade, Du, and Wang (2008) suggests that when students are asked to develop their own rubric, their subsequent performance improves.

Rubrics can be especially useful in providing teachers with guidance when attempting to evaluate portfolios or other kinds of performance, a task that is often alarmingly subjective. Assessment in accordance with clearly defined criteria makes the assigning of grades or of evaluative comments simpler, more transparent, more consistent, and fairer (Kan & Bulut, 2014). In fact, a well-designed rubric can even be used by students to rank their own work or that of other students.

Although rubrics vary considerably in detail and form, most share the following characteris- tics (Brown, 2008):

• They specify in writing the final objective or standard that serves as a goal for an instructional sequence and that will be evident in a performance or product.

• They include a scale that represents a ranking of behaviors, performances, skills, or products that can be used in assessing the extent to which student performance matches the final goal.

C A S E S F R O M T H E C L A S S R O O M : M Y L A M E E X C U S E The Setting: Carolyn White’s eighth-grade class

The Class Project: Friday afternoon essay and portfolio time

Carolyn White: It’s new and old essay time.

Keira: Good! (The sentiment is echoed by many others.)

Ms. White: Not a single essay was missing last week. They’re all here and they’re good. I mean awesome! So now I’ll trade you last week’s essay for your new one.

(In this English class, every student writes a weekly essay on a topic brainstormed by the class—topics such as: “My lame excuse,” “Why I hate cell phones,” “The lost memory” . . .)

Ms. White: Who is sharing their essay this week?

Robert: Me. And Keira and Evey.

Ms. White: Good. Go ahead, Robert.

My Lame Excuse, Robert reads. This essay is not half as good as the one I wrote. It’s a cheap replacement. You see, I didn’t need an excuse for the first essay I wrote because it was brilliant. My mom said so. It was about a new discovery in plasma physics. Why didn’t I hand it in, you ask? Well that’s a long story . . .

(And the essay continues.)

Lef83909_10_ch10_309-342.indd 330 12/14/17 4:24 PM

331

Section 10.5 Performance-Based Assessments

The best use of rubrics, note Siegel and associates (2011), is not so much for scoring students’ work but rather as a tool for improving learning and instruction. Rubrics clarify instructional goals, provide feedback, and improve self-evaluation. Figure 10.6 presents a very general rubric that might be used to assess progress in learning to speak a second language.

Figure 10.6: Example of a rubric ሁ Part of a rubric relating to learning conversational Spanish in high school. Rubrics can be very

useful for clarifying and communicating instructional goals, and also as a means of providing feedback to learners and promoting self-evaluation.

D ev

el op

in g

Expanding vocabulary; uses simple past and future tense; conversational skills limited to brief questions and responses

B eg

in ni

G oa

l: B

as ic

fl ue

nc y

in S

pa ni

Limited vocabulary; simple nouns; basic verbs; uses present tense exclusively A

cc om

pl is

he d Vocabularyadequate for

day-to-day conversations; correct uses of imperfect and simple past tense; clear pronunciation

Ex em

pl ar

Initiates conversation; chooses to read Spanish novels or newspapers; correct use of subjunctive tense

Ev al

ua tio

Evaluation of Performance Assessment In many ways, our emphases on learning how to learn and on autonomous, reflective, inde- pendent, creative thinking are incompatible with current testing practices. To the extent that our assessments demand the sort of linear thinking that is evidenced by acceptance and reproduction of single correct answers, they foster a type of teaching and learning that does not result in minds that work well but rather in minds that reproduce well.

Performance-based assessment appears to have several distinct advantages (Gronlund & Waugh, 2013; Newhouse, 2011):

• It is better attuned to our changing educational emphases. • It can be more equitable than more traditional approaches. • It provides additional ways of viewing competence and even excellence. • It might expose social and intellectual skills that more traditional measures would

overlook. • It bases assessment on a wider range of evidence than has been customary. • To the extent that it is more “authentic,” it might be more relevant to competency in

the real world. • It can provide greater motivation for students by clarifying goals and making learn-

ing more meaningful. • It encourages the application of learning to real-life situations.

But performance-based assessment is clearly more cumbersome, more time-consuming, more difficult, and in most cases less exact. As a result, it is not often used for very large-scale assessments. Also, performance-based assessments are not as easily quantifiable as more objective measures and are therefore not as useful for educational decisions that require com- parison. In short, they provide a less certain basis for answering questions such as “Which 120 of these 500 college applicants should be admitted?” or “Which one of these 120 should be given the Earnest Q. Honest Memorial Scholarship?”

Lef83909_10_ch10_309-342.indd 331 12/14/17 4:24 PM

332

Section 10.6 Reporting to Students and Parents

10.6 Reporting to Students and Parents Both summative and formative evaluation require that the results of assessments be com- municated to students and parents. The feedback that students receive about their learning can be tremendously valuable in guiding and motivating their efforts. It can also be highly reinforcing in this achievement-oriented society.

Grades Grades are probably still the most common way of summarizing achievement and progress. Grades might be expressed as letters, percentages, percentiles, or some other score that has clear meaning. As we saw earlier, however, teachers now often use portfolios and develop- mental checklists instead of grades, especially for younger learners.

The grades you assign as a teacher will typically be derived from a number of different sources: scores on tests and quizzes; marks given for homework and written assignments; marks given for special projects; and assessments of in-class work. You might also want to give some consideration to participation in class as well as to general behavior and effort. Deciding what is the fairest and most accurate way of calculating grades—that is, determin- ing what sort of weighting you should assign to this and to that, and what you should do about missing assignments and missing students—is not a simple and easy task. But no matter what you decide, no student’s grade should come as a surprise. Your criteria for grading, and your ongoing use of formative assessment, should make things so clear that no one is shocked when they first look at their report card.

Report Cards Report cards might be “hard copy” printed and handwritten documents. They might also be available online in the form of e-report cards. Software for e-report cards is widely available. In addition to grades, teacher comments, and developmental assessments, electronic report cards can include a wide variety of information such as attendance records, assignments, extracurricular activi- ties, and so on. Parents and students can access report cards by means of secure iden- tification and passwords, and can print them out if they wish. And, of course, those who don’t have access to the Internet can still obtain printed copies from their schools.

Although report cards are usually provided for students at the end of each school term, there are times when teachers might give children report cards more often—sometimes even on a daily basis. This might be the case, for example, for children who are placed on reinforcement programs where some, or even all, of the reinforcement is provided by parents. The daily report card serves to inform parents about the child’s behavior and accomplishments while at school.

Todd Warnock/DigitalVision/Thinkstock ሁ Schools are making increasing use of e-report

cards that parents and students can print out or access online.

Lef83909_10_ch10_309-342.indd 332 12/14/17 4:24 PM

333

Section 10.7 High-Stakes Testing

Parent–Teacher Conferences Electronic and hardcopy report cards are usually accompanied by parent–teacher confer- ences. These conferences give parents an opportunity to become more closely involved in the education of their children. Especially in the lower grades, scheduled parent–teacher confer- ences are often supplemented with more informal parent–teacher interaction.

10.7 High-Stakes Testing Recent times have brought some dramatic changes in the purposes of assessment in schools. With the passage of the No Child Left Behind (NCLB) act—later replaced by the Every Student Succeeds Act (ESSA)—and the introduction of the Race to the Top (R2T) educational fund- ing program, a new era was ushered in—the era of high-stakes testing—testing where the results have potentially dramatic implications.

The basic principle upon which the NCLB act was founded was that of accountability (Walker, 2015). That is, the act sought to hold schools accountable for important goals that include the requirement that all students reach certain standards in reading and mathematics, that all be taught by highly qualified teachers, and that all become proficient in English. Related to this, the Race to the Top Program was designed to reward schools that reached certain standards (Race to the Top Fund, 2016).

How can you tell that all students have reached the required standards? That teachers are highly qualified? That all learners have reached a high level of proficiency in English?

The simple answer is, by using tests. And this is where the expression high-stakes testing gets its meaning. High-stakes tests are those that can have significant consequences for teach- ers, students, and schools—consequences such as those relating to selection of students for programs, teacher certification, college admission, graduation, and determining school accountability.

Following the passage of the NCLB act, tests that assess the proficiency of students had very high stakes for schools because federal funds were often tied directly to performance on those tests. For example, the NCLB act required each state to set specific levels of proficiency as yearly targets with a view to reaching the objective of having every child reach high lev- els of proficiency in mathematics and reading. The consequences of not reaching the yearly targets—that is, of not making what was deemed adequate yearly progress (AYP)—were spelled out by the act. Schools that didn’t reach AYP targets two consecutive years were des- ignated as schools in need of improvement, and were required to provide supplemental ser- vices for students. A school that failed to make AYP for four or more years could be required to replace staff, to hire outside experts, or even to be run by outside administrators.

Lower-Stakes Testing With the passage of ESSA, AYP requirements are now history. Although schools are still expected to provide evidence of accountability and to reach certain standards, they are no lon- ger subject to the high-stakes testing requirements or punitive measures that marked NCLB (Walker, 2015). Schools are still required to administer annual tests in grades 3 through 8 and

Lef83909_10_ch10_309-342.indd 333 12/14/17 4:24 PM

334

Section 10.7 High-Stakes Testing

once more in high school. However, funding is now provided to improve tests and testing pro- cedures, and the goal of mandatory testing is more the improvement of teaching and learning than the determination of teacher and school accountability. In addition, barring state laws to the contrary, parents have the right to have their children opt out of statewide assessments.

As a result of the changes brought about by ESSA, statewide tests are generally not accompa- nied by stakes nearly as high as was the case under NCLB. But many instances of student and teacher assessments are still relatively high stakes. For example, tests of teacher competence can have very high stakes for teachers, or for those who would like to be teachers. The most commonly used tests for assessing teacher competence are the Praxis® tests—now used in a large number of states (About the Praxis® Tests, 2016). These are based on a list of stan- dards describing the skills, attitudes, and knowledge required of beginning teachers. This list, known as the InTASC standards (presented in Chapter 1, Table 1.1), was developed by a group of educators representing a consortium of states. One form of the Praxis® series of tests is designed to be used before admission to teacher training institutions (the Praxis® Core Aca- demic Skills for Educators [Core]); another is intended to provide information about subject matter knowledge, as well as specific teaching skills, and may be used for teacher certification (Praxis® Subject Assessments). Although these tests are not intended to be used for making hiring or firing decisions, or to rank teachers, their use can have important consequences for admission and certification. As such, they can be very high-stakes tests.

The Controversy High-stakes (and even lower-stakes) state or federally mandated testing programs have not been without controversy. True, most educators applaud the main goal of these initiatives, which is to ensure that the highest possible quality of education be provided for all learners regardless of their abilities, backgrounds, or zip codes. And most would agree that teachers and administrators should be held accountable for what happens in their schools.

Pros Some of the consequences of high-stakes testing are clearly positive. Most importantly, despite their weaknesses and misuses, standardized tests are more likely to be valid, reliable, objec- tive, and fair than other forms of evidence upon which teachers and educators base their judg- ments. Often, too, high-stakes testing can lead to an increase in time spent in teaching and learning, increased effort on the part of both teachers and learners, and more effective learning.

There is also evidence that high-stakes testing can serve to motivate students and teachers and make them work harder, explains Hamilton (2008). Mandatory testing can play an important role in making sure that certain academic standards are met, and it might move teachers to improve their instructional procedures. In addition, standardized achievement tests can assist teachers in creating specific learning plans for individual students, and can encourage learn- ers to focus on subjects that are important. Also, the results of high-stakes tests are typically communicated to parents, often involving them more closely in the education of their children.

Lef83909_10_ch10_309-342.indd 334 12/14/17 4:24 PM

335

Chapter 10 Summary and Resources

Cons But there are also negatives associated with high-stakes tests. There is little evidence, report Dianis, Jackson, and Noguera (2015), that high-stakes testing has resulted in widespread academic improvement. For one thing, note these authors, state or federally mandated high- stakes tests do nothing to bring disadvantaged learners out of poverty.

There is also evidence that the information derived from high-stakes testing can sometimes provide a distorted view of student achievement and proficiency. This is especially true when teachers have felt compelled to direct their instruction to the test itself, sometimes even coaching students on similar test items. Some research shows, for example, that you can even increase scores on intelligence tests by an average of 7 to 15 points by carefully preparing test-takers beforehand (Bloemers, 2007). Unfortunately, however, teaching-to-the-test has some negative effects. These include a “dumbing down” and a “narrowing” of the curriculum.

Valli, Croninger, and Walters (2007) also suggest that the demand for teacher accountability and accompanying high-stakes testing focuses too much attention on student performance on high-stakes tests and can easily undermine high-quality teaching and learning. Too often, raising educational achievement is equated with raising test scores.

For these and a variety of other reasons, the act that replaced the NCLB, the Every Student Succeeds Act (ESSA), has moved significantly from the era of high-stakes testing toward a less intrusive and less punitive model.

But we have certainly not heard the last word.

Chapter 10 Summary and Resources

Key Points • Measurement is the use of an instrument to gauge the quantity of a property or

behavior. Evaluation involves making a decision about quality, goodness, or appro- priateness. Assessment includes aspects of both. Summative evaluation involves the use of assessment primarily to summarize and assign a number to what students have accomplished; formative evaluation is geared toward improving teaching and learning.

• Educational goals can be broad statements relating to the general purpose of educa- tion. To be useful, instructional objectives should specify both what the learner must do and the criteria of acceptable performance. They can be highly performance- oriented and specific (Mager) or both general and specific (Gronlund).

• A test blueprint specifies topics or behaviors to be sampled as well as the number or proportion of items that will relate to each.

Lef83909_10_ch10_309-342.indd 335 12/14/17 4:25 PM

336

Chapter 10 Summary and Resources

• Good measuring instruments need to be fair, valid (measure what they purport to measure), and reliable (measure consistently). Aspects of validity include face valid- ity, content validity, and predictive validity.

• Reliability can be measured by looking at correlations between repeated presenta- tions of the same test (repeated-measures reliability), between different forms of the same test (parallel-forms reliability), or between halves of a single test (split-half reliability).

• Teacher-made paper-and-pencil tests are either objective (true–false, supply-type, matching, or multiple-choice) or essay. Essay tests are better for tapping higher mental processes, allow for more divergence, and are less time-consuming to pre- pare; they are also more limited in content, less reliable, and more time-consuming to score.

• Good essay examinations should sample processes not easily measured with objec- tive tests; be specific and clearly worded; allow sufficient time for answering; be directed toward instructional goals; and specify relative weightings of different questions. Good multiple-choice items have clear, meaningful stems; distractors of approximately equal plausibility; and no double negatives, absolute qualifiers (always, never), or other unintentional cues (a, an, singulars, plurals).

• One way to give meaning to raw scores on achievement tests is to convert them to percentage scores, percentiles, or letter grades. Measures of central tendency like the mean, the mode, and the median may also be useful.

• Schools have traditionally used norm-referenced tests (judging individual perfor- mance in relation to the performance of other students). Criterion-referenced tests compare individual performance to pre-established criteria rather than to the per- formance of other students.

• One alternative to traditional testing is performance-based assessment, which might include developmental assessments, demonstrations, exhibitions, portfolios, or rubrics as guides for instruction and evaluation.

• Summative evaluations are typically communicated to both students and parents, often as grades in report cards (electronic or hardcopy) as well as through parent– teacher conferences.

• Standardized tests are professionally developed instruments—usually measures of intelligence, personality, or achievement—that provide norms or standards for judging individual performance. They often provide age-equivalent scores, grade- equivalent scores, or percentiles.

• High-stakes tests are tests with potentially serious consequences. Standardized achievement tests, sometimes mandated by national or state legislation and used to assess the extent to which students and schools have reached required levels of proficiency, are high-stakes tests.

• Critics of high-stakes testing point to the possibility that their use places too much emphasis on achievement in a narrow sense and encourages schools to teach to the tests, resulting in the neglect of other important areas of learning and development.

Lef83909_10_ch10_309-342.indd 336 12/14/17 4:25 PM

337

Chapter 10 Summary and Resources

Posttest

1. The best example of an educational goal is a. Students will appreciate classical music. b. Students will solve single-digit division problems. c. Students will compose original haikus. d. Students will create a timeline for World War II.

2. Lucian picks up his test for his history class. As he looks it over, he thinks, “This does not look like the correct history test.” What type of validity is Lucian evaluating? a. Content validity b. Predictive validity c. Face validity d. Concurrent validity

3. Kate’s parents want to meet with her teacher to discuss her standardized test scores. She received a raw score of 32 and her percentile rank was 94. Which of the follow- ing statements is the MOST accurate interpretation for her teacher to make? a. 94% in the comparison group answered 32 questions correctly. b. Kate correctly answered 32% of the questions. c. Kate correctly answered 94% of the questions. d. 6% in the comparison group received raw scores higher than 32.

4. Which of the following measures of central tendency is correctly defined? a. Mean—midpoint or score at 50th percentile b. Median—most common score c. Mode—most popular score d. Medium—score at halfway point

5. Performance-based collections of student work that often help to improve perfor- mance and motivation are called a. demonstrations. b. portfolios. c. exhibitions. d. rubrics.

6. Which of the following considerations in grade assignment tends to be the most subjective? a. Participation b. Attendance c. Exam marks d. Homework scores

Answers: 1(a), 2(c), 3(d), 4(c), 5(b), 6(a)

Lef83909_10_ch10_309-342.indd 337 12/14/17 4:25 PM

338

Chapter 10 Summary and Resources

Critical Thinking Exercises • How might measurement and evaluation be involved in the assessment of student

performance in the classroom? Provide examples. • Write instructional objectives for a simple lesson to illustrate the approaches of

Mager and Gronlund. How are the objectives different? Are there any similarities? • How might you know that your tests are reliable and that they illustrate each of the

following kinds of validity: face, content, predictive? Explain your reasoning. • Evaluate the relative merits of criterion-referenced and norm-referenced testing.

When would one test be more appropriate to use than the other? Provide examples. • What are portfolios, and how might they be used to evaluate student performance? • If you were to debate the systemwide use of high-stakes tests, what would be some

pros and cons?

Web Resources For more information about reasons why some educators argue against the use of standard- ized tests, visit:

http://institute4learning.com/blog/2013/02/28/15-reasons-why-standardized -tests-are-worthless-2/

For a more positive view of standardized testing in schools, visit:

http://educationpost.org/dont-believe-the-hype-standardized-tests-are-good-for -children-families-and-schools/

For a comparison of school-based, norm-referenced, and criterion-referenced testing, visit:

https://prezi.com/_tuehshw9yqj/norm-referenced-versus-criterion-referenced -assessment/

For more information on high-stakes testing, visit:

http://edglossary.org/high-stakes-testing/

Answers to Pretest 1. True. Assessment determines student performance and can include measurement

and evaluation. 2. False. Making tests longer does improve reliability; however, having moderately dif-

ficult items instead of very difficult ones improves reliability. 3. True. Scoring the same test item for all students is one method to objectively score

tests. Other methods include being consistently objective and using a rubric. 4. False. Criterion-referenced testing is used to identify if students have reached mas-

tery level and does so without comparing students to others. 5. True. Performance assessments focus on students’ understanding and their ability

to apply knowledge to topics or situations that traditional assessments might be limited in measuring.

6. True. High-stakes testing can limit the curriculum taught to students and decrease the quality of teaching and learning.

Lef83909_10_ch10_309-342.indd 338 12/14/17 4:25 PM

http://institute4learning.com/blog/2013/02/28/15-reasons-why-standardized-tests-are-worthless-2/

http://educationpost.org/dont-believe-the-hype-standardized-tests-are-good-for-children-families-and-schools/

https://prezi.com/_tuehshw9yqj/norm-referenced-versus-criterion-referenced-assessment/

http://edglossary.org/high-stakes-testing/

339

Chapter 10 Summary and Resources

Answers to Posttest 1. Educational goals can be broad. They determine what needs to be taught and which

behaviors to evaluate. 2. Face validity describes how a test appears to appropriately measure the topic. Glanc-

ing through the history test, Lucian should see relevant questions regarding histori- cal events he learned in class, the text, and assignments.

3. Percentiles describe the cases that are at or below a specific percentage of cases. In this example, 94% of students received a raw score equal to or less than 32, meaning 6% of students scored above a raw score of 32.

4. The mean is the average, the median is the midpoint, and the medium is not related to measuring central tendency.

5. Portfolios offer a collection of a student’s work and performance over time. Research shows that portfolios receive a more positive response from students than tradi- tional assessment types and can increase motivation.

6. Participation and behavior are factors that can determine students’ grades. Grades are determined from assessment scores, home and in-class assignments, and stu- dents’ level of participation and behavior. The teacher should decide how much weight these sources will factor into students’ grades.

Key Terms adequate yearly progress (AYP) Expres- sion for the measurement of the annual progress schools and school districts were making with respect to mathematics and reading proficiency targets mandated by the No Child Left Behind act. Following the pas- sage of the Every Student Succeeds Act (ESSA) in 2015, AYP measures are largely historical.

age-equivalent scores Scores converted to age equivalents using standardized test norms. See also grade-equivalent score, stan- dardized test.

assessment A general term for the process of appraising student performance. See also evaluation, measurement.

authentic assessment Refers to assess- ment procedures designed to allow students to demonstrate their ability to apply learn- ing in real-life situations. See also perfor- mance assessment.

behavioral objective An instructional objective expressed in terms of specific, observable, measurable behaviors.

central tendency The tendency for the majority of scores in a normal distribution to cluster around the center of the distribu- tion. Measures of central tendency include the mean, the median, and the mode. See also mean, median, mode.

content validity Test validity determined by a careful analysis of the content of test items and a comparison of this content with course objectives. See also face validity, pre- dictive validity.

criterion-referenced testing A test in which the student is judged relative to a criterion rather than relative to the perfor- mance of other students. See also norm- referenced testing.

demonstrations An approach to performance-based assessment; students are given an opportunity to perform real-life activities that require and illustrate the use of the skills and cognitive processes being assessed.

Lef83909_10_ch10_309-342.indd 339 12/14/17 4:25 PM

340

Chapter 10 Summary and Resources

developmental assessments Performance-based assessments that look at the student’s performance in relation to developmental or course-based expecta- tions rather than in relation to the perfor- mance of other students. Often uses check- lists of expected (desired) behaviors.

educational goals Intended or desired outcomes of the educational process. Often expressed in terms of instructional objec- tives that can range from highly general to very specific. See also instructional objectives.

e-report cards Report cards available online rather than in printed and handwrit- ten form, accessible by means of a password.

essay tests Tests that require testees to construct responses of varying lengths in sentence, paragraph, or essay form. See also objective tests.

evaluation In contrast to measurement, evaluation involves making a value judg- ment—deciding on the goodness or badness of performance. See also measurement.

exhibitions Performance-based assess- ment procedures that require the public display of competence.

face validity The extent to which a test appears to be measuring what it is intended to measure. See also content validity, predic- tive validity, reliability, validity.

formative evaluation Evaluation under- taken before and during instruction, designed primarily to assist the learner by identifying strengths and weaknesses. See also summative evaluation.

grade-equivalent scores Scores that have been converted to a grade equivalent using standardized test norms. See also age- equivalent scores, standardized tests.

high-stakes testing In education, typically a standardized test upon whose outcomes much depends—for example, a career, pro- motion, college entrance, continued funding, or dismissal.

instructional objectives Goals or intended results of instruction. Also termed behav- ioral objective.

mean The arithmetic average of a set of scores. See also central tendency, median, mode.

measurement The application of an instru- ment to gauge the quantity of something, as opposed to its quality. Assessing quality involves evaluation, not measurement. See also assessment, evaluation.

median The midpoint or 50th percentile of a distribution; the point at or below which 50 percent of all scores fall. See also central tendency, mean, mode.

mode The most frequently occurring score(s) in a distribution. See also central tendency, mean, median.

norm-referenced testing A test where the student is competing against the per- formance of other students rather than in relation to a pre-established criterion of acceptable performance. See also criterion- referenced testing.

objective tests A label used for tests in which the scoring procedure is simple, clear, and objective. Includes multiple-choice, completion, matching, and true–false for- mats. See also essay tests.

parallel-forms reliability A measure of test consistency (reliability) obtained by looking at the correlation between scores obtained by the same individual on two dif- ferent but equivalent (parallel) forms of one test. See also split-half reliability.

Lef83909_10_ch10_309-342.indd 340 12/18/17 1:42 PM

341

Chapter 10 Summary and Resources

percentile The point at or below which a specified percentage of scores fall.

performance assessment Assessment that looks at the actual performance of students in situations as close to real life as possible. See also authentic assessment.

portfolios In educational assessment, collections of actual samples of students’ performance and achievements.

predictive validity A measure of the extent to which predictions based on test results are accurate. See also content validity, face validity.

raw scores The actual numerical scores that a testee obtains on tests; the testee’s scores before they are converted to grade- or age-equivalent scores, Intelligence Quotients (IQ), or other norms.

repeated-measures reliability An esti- mate of the consistency (reliability) of a test based on the degree of agreement among scores obtained from different presenta- tions of the same test. See also parallel-forms reliability.

rubric A written set of guidelines used for assessing the quality of a performance or product.

split-half reliability An index of test reli- ability (consistency) derived by arbitrarily dividing a test into parallel halves (odd- and even-numbered items, for example) and looking at the agreement between the two scores obtained by each individual on the two halves. See also parallel-forms reliability.

standardized achievement tests Profes- sionally developed and normed tests that are designed to measure achievement and to provide some basis for judging the rela- tive quality of that achievement, given the student’s age and grade placement. See also standardized tests.

standardized tests Professionally developed—rather than teacher-made— tests that provide the user with norms (standards) and typically indicates the average or expected performance of groups of subjects of certain grades or ages. See also age-equivalent scores, grade- equivalent scores.

summative evaluation The type of evalu- ation that occurs at the end of an instruc- tional sequence and that is designed primar- ily to provide a grade. See also formative evaluation.

teacher-made tests Any of the wide variety of tests written, developed, or organized by teachers, usually for the purpose of evaluat- ing students or assessing the effectiveness of instruction. See also standardized tests.

test blueprint A table of specifications for a teacher-made test.

ሁ Purring among suckling young has been reported in the black bear. Growling among adult bears has also been reported (Ewer, 1973).

Lef83909_10_ch10_309-342.indd 341 12/14/17 4:25 PM

Lef83909_10_ch10_309-342.indd 342 12/14/17 4:25 PM