EDU 530 Week 1 Discussion

209

Chapter 8

Performance Assessment

Chief Chapter Outcome

A sufficiently broad understanding of performance assessment to distinguish between accurate and inaccurate statements regarding the nature of performance tests, the identification of tasks suitable for such tests, and the scoring of students’ performances

Learning Objectives

8.1 Identify three defining characteristics of performance tests intended to maximize their contributions to teachers’ instructional effectiveness and describe how they should be applied.

8.2 Isolate the substantive distinctions among three different procedures for evaluating the quality of students’ responses to performance assessments, that is, via task-specific, hypergeneral, and skill-focused rubrics.

During the early 1990s, a good many educational policymakers became enamored of performance assessment, which is an approach to measuring a student’s status based on the way the student completes a specified task. Theoretically, of course, when the student chooses between true and false for a binary-choice item, the student is completing a task, although an obviously modest one. But the proponents of performance assessment have measure- ment schemes in mind that are meaningfully different from binary-choice and multiple-choice tests. Indeed, it was dissatisfaction with traditional paper- and-pencil tests that caused many educators to travel with enthusiasm down the performance-testing trail.

M08_POPH0936_10_SE_C08.indd 209M08_POPH0936_10_SE_C08.indd 209 09/11/23 6:21 PM09/11/23 6:21 PM

210 Chapter 8 performance assessment

What Is a Performance Test? Before digging into what makes performance tests tick and how you might use them in your own classroom, we’d best explore the chief attributes of such an assessment approach. Even though all educational tests require students to per- form in some way, when most educators talk about performance tests, they are thinking about assessments in which the student is required to construct an origi- nal response. It may be useful for you to regard performance tests as an assess- ment task in which the students make products or engage in behaviors other than answering selected-response or constructed-response items. Often, an examiner (such as the teacher) observes the process of construction, in which case observa- tion of the student’s performance and judgment about the quality of that perfor- mance are required. Frequently, performance tests call for students to generate some sort of specified product whose quality can then be evaluated.

More than a half-century ago, Fitzpatrick and Morrison (1971) observed that “there is no absolute distinction between performance tests and other classes of tests.” They pointed out that the distinction between performance assessments and more conventional tests is chiefly the degree to which the examination simu- lates the real-world criterion situation—that is, the extent to which the examina- tion calls for student behaviors approximating those about which we wish to make inferences.

Suppose, for example, a teacher who had been instructing students in the process of collaborative problem solving wanted to see whether students had acquired this collaborative skill. The inference at issue centers on the extent to which each student has mastered the skill. The educational decision on the line might be whether particular students need additional instruction or, on the con- trary, it’s time to move on to other, possibly more advanced, curricular aims. The teacher’s real interest, then, is in how well students can work with other students to arrive collaboratively at solutions to problems. In Figure 8.1, you will see several assessment procedures that could be used to get a fix on a student’s collaborative problem-solving skills. Note that the two selected-response assess- ment options (numbers 1 and 2) don’t really ask students to construct anything. For the other three constructed-response assessment options (numbers 3, 4, and 5), however, there are clear differences in the degree to which the task presented to the student coincides with the class of tasks called for by the teacher’s curricular aim. Assessment Option 5, for example, is obviously the closest match to the behavior called for in the curricular aim. Yet, Assessment Option 4 is surely more of a “performance test” than is Assessment Option 1.

It should be apparent to you, then, that different educators will be using the phrase performance assessment to refer to very different kinds of assessment approaches. Many teachers, for example, are willing to consider short-answer and essay tests as a form of performance assessment. In other words, those teach- ers essentially equate performance assessment with any form of constructed- response assessment. Other teachers establish more stringent requirements for a

M08_POPH0936_10_SE_C08.indd 210M08_POPH0936_10_SE_C08.indd 210 09/11/23 6:21 PM09/11/23 6:21 PM

­Wat s a performance testt 211

measurement procedure to be accurately described as a performance assessment. For example, some performance-assessment proponents contend that genuine performance assessments must exhibit at least three features:

• Multiple evaluative criteria. The student’s performance must be judged using more than one evaluative criterion. To illustrate, a student’s ability to speak Spanish might be appraised on the basis of the student’s accent, syntax, and vocabulary.

• Prespecified quality standards. Each of the evaluative criteria on which a stu- dent’s performance is to be judged is clearly explicated in advance of judging the quality of the student’s performance.

• Judgmental appraisal. Unlike the scoring of selected-response tests in which electronic computers and scanning machines can, once programmed, carry on without the need of humankind, genuine performance assessments depend on human judgments to determine how acceptable a student’s performance really is.

Looking back to Figure 8.1, it is clear that if the foregoing three requirements were applied to the five assessment options, then Assessment Option 5 would qualify as a performance test, and Assessment Option 4 probably would as well,

Assessment Options

5. Students work in small groups to solve previously unencountered problems. Teacher observes and judges their efforts.

4. Students are given a new problem, then asked to write an essay regarding how a group should go about solving it.

3. Students are asked a series of questions regarding ways of solving problems collaboratively, then asked to supply short answers to the questions.

2. Students answer a series of multiple-choice tests about the next steps to take when solving problems in groups.

1. Students respond to true–false questions about the best procedures to follow in group problem solving.

Students can solve problems collaboratively.

Curricular Aim

Figure 8.1 a Set of assessment Options tWat Vary in tWe Degree to ­WicW a Student’s task approximates tWe Curricularly targeted BeWavior

M08_POPH0936_10_SE_C08.indd 211M08_POPH0936_10_SE_C08.indd 211 09/11/23 6:21 PM09/11/23 6:21 PM

212 Chapter 8 performance assessment

but the other three assessment options wouldn’t qualify under a definition of performance assessment requiring the incorporation of multiple evaluative criteria, prespecified quality standards, and judgmental appraisals.

A good many advocates of performance assessment would prefer that the tasks presented to students represent real-world rather than school-world kinds of problems. Other proponents of performance assessment would be elated simply if more school-world measurement was constructed response rather than selected response in nature. Still other advocates of performance testing want the tasks in performance tests to be genuinely demanding—that is, way up the ladder of cognitive difficulty. In short, proponents of performance assessment often advo- cate somewhat different approaches to measuring students on the basis of those students’ performances.

In light of the astonishing advances we now see every few months in the sorts of computer-delivered stimuli for various kinds of assessment—performance tests surely included—the potential nature of performance-test tasks seems practi- cally unlimited. For example, the possibility of digitally simulating a variety of authentic performance-test tasks provides developers of performance tests with an ever-increasing range of powerful performance assessments placing students in a test-generated “virtual world.”

You’ll sometimes encounter educators who use other phrases to describe performance assessment. For example, they may use the label authentic assessment (because the assessment tasks more closely coincide with real- life, nonschool tasks) or the label alternative assessment (because such assess- ments constitute an alternative to traditional, paper-and-pencil tests). In the next chapter, we’ll be considering portfolio assessment, which is a particular type of performance assessment and should not be considered a synony- mous descriptor for the performance-assessment approach to educational measurement.

To splash a bit of reality juice on this chapter, it may be helpful for you to recognize a real-world fact about educational performance assessment. Here it goes: Although most educators regard performance testing as an effective way to measure students’ mastery of important skills instead of using many traditional testing tactics, in recent years the financial demands of such testing have ren- dered them nonexistent or, at best, tokenistic, in many settings. Yes, as the chapter probes the innards of this sort of educational testing, you will discover that it embodies some serious advantages, particularly its contributions to instruction. Yet, you will also see that a full-fledged reliance on performance testing for many students, such as we see in states’ annual accountability tests, often renders the widespread use of performance tests prohibitively expensive. More affordable, of course, is teachers’ use of this potent assessment approach for their own class- room assessments.

We now turn to the twin issues that are at the heart of performance assess- ments: selecting appropriate tasks for students and, once the students have tackled those tasks, judging the adequacy of students’ responses.

M08_POPH0936_10_SE_C08.indd 212M08_POPH0936_10_SE_C08.indd 212 09/11/23 6:21 PM09/11/23 6:21 PM

dentifying Suitable tasks for performance assessment 213

Identifying Suitable Tasks for Performance Assessment Performance assessment typically requires students to respond to a small number of more significant tasks, rather than to a large number of less significant tasks. Thus, rather than answering 50 multiple-choice items on a conventional chemistry examination, students who are being assessed via performance tasks may find themselves asked to perform an actual experiment in their chemistry class, and then prepare a written interpretation of the experiment’s results—along with an analytic critique of the procedures they used. From the chemistry teacher’s per- spective, instead of seeing how students respond to the 50 “ mini-tasks” represented in the multiple-choice test, an estimate of each student’s status must be derived from the student’s response to a single, complex task. Given the significance of each task used in a performance-testing approach to classroom assessment, it is apparent that great care must be taken in the selection of performance-assessment tasks. Generally speaking, classroom teachers will either have to (1) generate their own performance-test tasks or (2) select performance-test tasks from the increasing number of tasks currently available from educators elsewhere.

M08_POPH0936_10_SE_C08.indd 213M08_POPH0936_10_SE_C08.indd 213 09/11/23 6:21 PM09/11/23 6:21 PM

214 Chapter 8 performance assessment

Decision time Grow, plants, Grow!

Sofia Esposito is a third-year biology teacher in Kennedy High School. Because she has been convinced by several of her colleagues that traditional paper-and-pencil examinations fail to capture the richness of the scientific experience, Sofia has decided to base most of her students’ grades on a semester-long performance test. As Sofia contemplates her new assessment plan, she decides that 90 percent of the students’ grades will stem from the quality of their responses to the performance test’s task; 10 percent of the grades will be linked to classroom participation and to a few short true–false quizzes administered throughout the semester.

The task embodied in Sofia’s performance test requires each student to design and conduct a 2-month experiment to study the growth of three identical plants under different conditions, and then prepare a formal scientific report describing the experiment. Although most of Sofia’s students carry out their experiments at home, several students use the shelves at the rear of the classroom for their experimental plants. A number of students vary the amount of light or the kind of light received by the different plants, but most students modify the

nutrients given to their plants. After a few weeks of the 2-month experimental period, all of Sofia’s students seem to be satisfactorily under way with their experiments.

Several of the more experienced teachers in the school, however, have expressed their reservations to Sofia about what they regard as “overbooking on a single assessment experience.” The teachers suggested to Sofia that she will be unable to draw defensible inferences about her students’ true mastery of biological skills and knowledge on the basis of a single performance test. They urged her to reduce dramatically the grading weight for the performance test so that, instead, additional grade-contributing exams can also be given to the students.

Other colleagues, however, believe Sofia’s performance-test approach is precisely what is needed in courses such as biology. They recommended that she “stay the course” and alter “not one word” of her new assessment strategy.

If you were Sofia, what would your decision be?

Inferences and Tasks Consistent with the frequently asserted message of this text about classroom assessment, the chief determinants of how you assess your students are (1) the inference—that is, the interpretation—you want to make about those students and (2) the decision that will be based on the inference. For example, sup- pose you’re a history teacher and you’ve spent a summer at a lakeside cabin meditating about curricular matters (which, in one lazy setting or another, is the public’s perception of how most teachers spend their summer vacations). After three months of heavy curricular thought, you have concluded that what you really want to teach your students is to apply historical lessons of the past to the solution of current and future problems, which, at least to some extent, parallel the problems of the past. You have decided to abandon your week-long, 1500-item true–false final examination, which your stronger

M08_POPH0936_10_SE_C08.indd 214M08_POPH0936_10_SE_C08.indd 214 09/11/23 6:21 PM09/11/23 6:21 PM

dentifying Suitable tasks for performance assessment 215

students refer to as a “measurement marathon” and your weaker students describe by using a rich, if earthy, vocabulary. Instead of using true–false items, you are now committed to a performance-assessment strategy, and you wish to select tasks for your performance tests. You want the performance test to help you infer how well your students can draw on the lessons of the past to illuminate their approach to current and/or future problems.

In Figure 8.2, you will see a graphical depiction of the relationships among (1) a teacher’s key curricular aim, (2) the inference that the teacher wishes to draw about each student, and (3) the tasks for a performance test intended to secure data to support the inference that the teacher wishes to make. As you will note, the teacher’s curricular aim provides the source for the inference. The assessment tasks yield the evidence needed for the teacher to arrive at defensible inferences regarding the extent to which students can solve current or future problems using historical lessons. To the degree that students have mastered the curricular aim, the teachers will reach a decision about how much more instruction, if any, is needed.

The Generalizability Dilemma One of the most serious difficulties with performance assessment is that, because students respond to fewer tasks than would be the case with conventional paper-and-pencil testing, it is often more difficult to generalize accurately about what skills and knowledge are possessed by the student. To illustrate, let’s say you’re trying to get a fix on your students’ ability to multiply pairs of double- digit numbers. If, because of your instructional priorities, you can devote only a half-hour to assessment purposes, you could require the students to respond to 20 such multiplication problems in the 30 minutes available. (That’s probably more problems than you’d need, but this example attempts to draw a vivid con- trast for you.) From a student’s responses to 20 multiplication problems, you can get a pretty fair idea about what kind of double-digit multiplier the student is. As a consequence of the student’s performance on a reasonable sample of items repre- senting the curricular aim, you can sensibly conclude that “Xavier really knows

Figure 8.2 relationsWips among a teacWer’s Key Curricular aim, tWe assessment-Based nference Derived from tWe aim, and tWe performance-assessment tasks providing evidence for tWe nference

E.g., Students are given a current/future problem, then

asked to solve it using historically derived

insights.

Student Responses to Performance

Tasks E.g., Students’ ability to illuminate current/future problems with relevant

historical lessons is inferred.

Inferred Student Status

E.g., Students can use historical lessons to solve current/future

problems.

Derived from

Evidence for Key

Curricular Aim

M08_POPH0936_10_SE_C08.indd 215M08_POPH0936_10_SE_C08.indd 215 09/11/23 6:21 PM09/11/23 6:21 PM

216 Chapter 8 performance assessment

how to multiply when facing those sorts of problems,” or “Fred really couldn’t multiply double-digit multiplication problems if his life depended on it.” It is because you have adequately sampled the kinds of student performance (about which you wish to make an inference) that you can confidently make inferences about your students’ abilities to solve similar sorts of multiplication problems.

With only a 30-minute assessment period available, however, if you moved to a more elaborate kind of performance test, you might only be able to have students respond to one big-bopper item. For example, if you presented a multiplication-focused mathematics problem involving the use of manipulatives, and wanted your students to derive an original solution and then describe it in writing, you’d be lucky if your students could finish the task in half an hour. Based on this single task, how confident would you be in making inferences about your students’ abilities to perform comparable multiplication tasks?

And this, as you now see, is the rub with performance testing. Because stu- dents respond to fewer tasks, the teacher is put in a trickier spot when it comes to deriving accurate interpretations about students’ abilities. If you use only one performance test, and a student does well on the test, does this mean the stu- dent really possesses the category of skills the test was designed to measure, or did the student just get lucky? On the other hand, if a student messes up on a single-performance test, does this signify that the student really doesn’t possess the assessed skill, or was there a feature in this particular performance test that misled the student who, given other tasks, might have performed marvelously?

As a classroom teacher, you’re faced with two horns of a classic measurement dilemma. Although performance tests often measure the kinds of student abilities you’d prefer to assess (because those abilities are in line with really worthwhile curric- ular aims), the inferences you make about students on the basis of their responses to performance tests must often be made with increased caution. As with many dilem- mas, there may be no perfect way to resolve this dilemma. But there is, at least, a way of dealing with the dilemma as sensibly as you can. In this instance, the solution strat- egy is to devote great care to the selection of the tasks embodied in your performance tests. Among the most important considerations in selecting such tasks is to choose tasks that optimize the likelihood of accurately generalizing about your students’ capabilities. If you really keep generalizability at the forefront of your thoughts when you select or construct performance-test tasks, you’ll be able to make the strongest possible performance-based inferences about your students’ capabilities.

Factors to Consider When Evaluating Performance-Test Tasks We’ve now looked at what many measurement specialists regard as the most important factor you can consider when judging potential tasks for performance assessments—generalizability. Let’s look at a set of seven such factors you might wish to consider, whether you select a performance-test task from existing tasks or create your own performance-test tasks anew.

M08_POPH0936_10_SE_C08.indd 216M08_POPH0936_10_SE_C08.indd 216 09/11/23 6:21 PM09/11/23 6:21 PM

dentifying Suitable tasks for performance assessment 217

Evaluative Criteria for Performance-Test Tasks

• Generalizability. Is there a high likelihood that the students’ performance on the task will generalize to comparable tasks?

• Authenticity. Is the task similar to what students might encounter in the real world, as opposed to encountering such a task only in school?

• Multiple foci. Does the task measure multiple instructional outcomes instead of only one?

• Teachability. Is the task one that students can become more proficient in as a consequence of a teacher’s instructional efforts?

• Fairness. Is the task fair to all students—that is, does the task avoid bias based on such personal characteristics as students’ gender, ethnicity, or socio- economic status?

• Feasibility. Is the task realistically implementable in relation to its cost, space, time, and equipment requirements?

• Scorability. Is the task likely to elicit student responses that can be reliably and accurately evaluated?

Whether you’re developing your own tasks for performance tests or select- ing such tasks from an existing collection, you may wish to apply some but not all the factors listed here. Some teachers try to apply all seven factors, although they occasionally dump the authenticity criterion or the multiple foci criterion. In some instances, for example, school tasks rather than real-world tasks might be suitable for the kinds of inferences a teacher wishes to reach, so the authenticity criterion may not be all that relevant. And even though it is often economically advantageous to measure more than one outcome at one time, particularly con- sidering the time and effort that goes into almost any performance test, there may be cases in which a single educational outcome is so important that it war- rants a solo performance test. More often than not, though, a really good task for a performance test will satisfy most, if not all, of the seven evaluative criteria presented here.

Performance Tests and Teacher Time Back in Chapter 1, you were promised that if you completed this text with a rea- sonable degree of attentiveness, you’d become a better teacher. Now, here comes another promise for you: This text promises to be honest about the measurement mysteries being dissected. And this brings us to an important consideration regarding performance testing. Put briefly, it takes time!

Think for a moment about the time that you, as the teacher, must devote to (1) the selection of suitable tasks, (2) the development of an appropriate scheme for scoring students’ responses, and (3) the actual scoring of students’ responses. Talk to any teacher who has already used many classroom performance tests, and you’ll learn that it requires a ton of time to use performance assessment.

M08_POPH0936_10_SE_C08.indd 217M08_POPH0936_10_SE_C08.indd 217 09/11/23 6:21 PM09/11/23 6:21 PM

218 Chapter 8 performance assessment

So here’s one additional factor you should throw into your decision making about performance assessment. It is the significance of the skill you’re using the performance test to assess. Because you’ll almost certainly have time for only a handful of such performance tests in your teaching, make sure that every per- formance test you use is linked to a truly significant skill you’re trying to have your students acquire. If performance assessments aren’t based on genuinely demanding skills, you’ll soon stop using them because, to be truthful, they’ll be far more trouble than they’re worth.

The Role of Rubrics Performance assessments invariably are based on constructed-response measure- ment procedures in which students generate rather than select their responses. Student-constructed responses must be scored, however, and it’s clearly much tougher to score constructed responses than to score selected responses. The scoring of constructed responses centers on the evaluative criteria one calls on to determine the adequacy of students’ responses. Let’s turn our attention now to the evaluative factors we employ to decide whether students’ responses to per- formance tests are splendid or shabby.

A criterion, according to most people’s understanding, is a standard on which a decision or judgment can be based. In the case of scoring students’ responses to a performance test’s task, you’re clearly trying to make a judgment regarding the adequacy of the student’s constructed response. The specific criteria to be used in making that judgment will obviously influence the way you score a student’s response. For instance, if you were scoring a student’s written composition on the basis of organization, word choice, and communicative clarity, you might arrive at very different scores than if you had scored the composition on the basis of spelling, punctuation, and grammar. The evaluative criteria used when scor- ing students’ responses to performance tests (or their responses to any kind of constructed-response item) really control the whole assessment game.

Referring to my previously mentioned and oft-whined compulsion to fall back on my 5 years of Latin studies in high school and college, I feel responsible for explaining that the Latin word criterion is singular and the Latin word criteria is plural. Unfortunately, so many of today’s educators mix up these two terms that I don’t even become distraught about it anymore. However, now that you know the difference, if you find any of your colleagues erroneously saying “the criteria is” or “the criterion were,” you have permission to display an ever so subtle, yet altogether condescending, smirk.

The scoring procedures for judging students’ responses to performance tests are usually referred to these days as scoring rubrics or, more simply, as rubrics. A rubric that’s used to score students’ responses to a performance assessment has, at minimum, three important features:

M08_POPH0936_10_SE_C08.indd 218M08_POPH0936_10_SE_C08.indd 218 09/11/23 6:21 PM09/11/23 6:21 PM

tWe role of rubrics 219

• Evaluative criteria. These are the factors to be used in determining the quality of a student’s response.

• Descriptions of qualitative differences for all evaluative criteria. For each evalua- tive criterion, a description must be supplied so that qualitative distinctions in students’ responses can be made using the criterion.

• An indication of whether a holistic or analytic scoring approach is to be used. The rubric must indicate whether the evaluative criteria are to be applied collec- tively in the form of holistic scoring, or on a criterion-by-criterion basis in the form of analytic scoring.

The identification of a rubric’s evaluative criteria is probably the most important task for rubric developers. If you’re creating a rubric for a performance test that you wish to use in your own classroom, be careful not to come up with a lengthy laundry list of evaluative criteria which a student’s response should satisfy. Many seasoned scorers of students’ performance tests believe that when you isolate more than three or four evaluative criteria per rubric, you’ve identified too many. If you find your- self facing more than a few evaluative criteria, simply rank all the criteria in order of importance, and then chop off those listed lower than the very most important ones.

The next job you’ll have is deciding how to describe in words what a student’s response must display in order to be judged wonderful or woeful. The level of descriptive detail you apply needs to work for you. Remember, you’re devising a scoring rubric for your own classroom, not for a statewide or national test. Reduce the aversiveness of the work by employing brief descriptors of quality differences that you can use when teaching and that, if you’re instructionally astute, your students can use as well. However, because for instructional purposes you will invariably want your students to understand the meaning of each evaluative criterion and its worth, give some thought to how effectively the different quality labels you choose and the descriptions of evaluative criteria you employ can be communicated to students.

Finally, you’ll need to decide whether you’ll make a single, overall judg- ment about a student’s response by considering all of the rubric’s evaluative cri- teria as an amalgam (holistic scoring) or, instead, award points to the response on a criterion-by-criterion basis (analytic scoring). The virtue of holistic scoring, of course, is it’s quicker to do. The downside of holistic scoring is that it fails to communicate to students, especially low-performing students, the nature of their shortcomings. Clearly, analytic scoring is more likely than holistic scoring to yield diagnostically pinpointed scoring and sensitive feedback. Some class- room teachers have attempted to garner the best of both worlds by scoring all responses holistically and then analytically rescoring (for feedback purposes) all of the responses made by low-performing students.

Because most performance assessments call for fairly complex responses from students, there will usually be more than one evaluative criterion employed to score students’ responses. For each of the evaluative criteria chosen, a numerical scale is typically employed so that, for each criterion, a student’s response might be assigned a specified number of points—for instance, 0 to 6 points. Usually,

M08_POPH0936_10_SE_C08.indd 219M08_POPH0936_10_SE_C08.indd 219 09/11/23 6:21 PM09/11/23 6:21 PM

220 Chapter 8 performance assessment

these scale points are accompanied by verbal descriptors, but sometimes they aren’t. For instance, in a 5-point scale, the following descriptors might be used: =5 Exemplary, =4 Superior, =3 Satisfactory, =2 Weak, =1 Inadequate.

If no verbal descriptors are used for each score point on the scale, a scheme such as the following might be employed:

Excellent Unsatisfactory

6 5 4 3 2 1 0

In some cases, the scoring scale for each criterion is not numerical; that is, it consists only of verbal descriptors such as “exemplary,” “adequate,” and so on. Although such verbal scales can be useful with particular types of performance tests, their disadvantage is that scores from multiple criteria often cannot be added together in order to produce a meaningful, diagnostically discernible, overall score.

The heart of the intellectual process in isolating suitable evaluative criteria is to get to the essence of what the most significant factors are that distinguish acceptable from unacceptable responses. In this assessment instance, as in many others, less is more. A few truly important criteria are preferable to a ton of trifling criteria. Go for the big ones! If you need help in deciding which criteria to employ in conjunction with a particular performance test, don’t be reluctant to ask col- leagues to toss in their ideas regarding what significant factors to use (for a given performance test) in order to discriminate between super and subpar responses.

In this chapter you’ll be presented with scoring rubrics that can serve as use- ful models for your own construction of such rubrics. In those illustrative scoring rubrics, you’ll see that care has been taken to isolate a small number of instruction- ally addressable evaluative criteria. The greatest payoff from a well-formed scoring rubric, of course, is in its contribution to a teacher’s improved instruction.

To give you a better idea about the kinds of tasks that might be used in a performance test, and the way you might score a student’s responses, let’s take a gander at illustrative tasks for a performance test. The tasks presented in Figure 8.3 are intended to assess students’ mastery of an oral communication skill.

Rubrics: The Wretched and the Rapturous Unfortunately, many educators believe a rubric is a rubric is a rubric. Not so. Rubrics differ dramatically in their instructional worth. We’ll now consider two sorts of rubrics that are sordid and one sort that’s super.

Task-Specific Rubrics Some rubrics are created so that their evaluative criteria are linked only to the particular task embodied in a specific performance test. These are called task-specific rubrics. Such a rubric does little to illuminate instructional decision

M08_POPH0936_10_SE_C08.indd 220M08_POPH0936_10_SE_C08.indd 220 09/11/23 6:21 PM09/11/23 6:21 PM

rubrics: tWe ­retcWed and tWe rapturous 221

An Oral Communication Performance Test

Introduction There are numerous kinds of speaking tasks students must perform in everyday life, both in school and out of school. This performance assess- ment focuses on some of these tasks—namely, describing objects, events, and experiences; explaining the steps in a sequence; providing information in an emergency; and persuading someone. In order to accomplish a speaking task, the speaker must formulate and transmit a message to a listener. This process involves deciding what needs to be said, organizing the message, adapting the message to the listener and situation, choosing language to convey the message and, finally, delivering the message. The effectiveness of the speaker may be rated according to how well the speaker meets the requirement of the task.

Sample Tasks Description Task: Think about your favorite class or extracurricular activity

program?) Emergency Task: Imagine you are home alone and you smell smoke. You call the fire department and I answer your call. Talk to me as if you were talking on the telephone. Tell me everything I would need to know to get help to you. (Talk directly to me; begin by saying hello.)

Sequence Task: Think about something you know how to cook. Explain to

Persuasion Task: Think about one change you would like to see made in your school, such as a change in rules or procedures. Imagine I am the principal of your school. Try to convince me the school should make this change. (How about something like a change in the rules about hall passes or the procedures for enrolling in courses?) Source: Based on assessment efforts of the Massachusetts Department of Education.

in school. Describe to me everything you can about it so I will know a lot about it. (How about something like a school subject, a club, or a sports

me, step by step, how to make it. (How about something like popcorn, a sandwich, or scrambled eggs?)

Figure 8.3 task Description and Sample tasks for a One-on-One Oral Communication performance assessment

making, because it implies that students’ performances on the constructed- response test’s specific task are what’s important. They’re not! What’s important is the student’s ability to perform well on the class of tasks that can be accom- plished by using the skill being measured by the assessment. If students are taught to shine only on the single task represented in a performance test, but not on the full range of comparable tasks, students lose out. If students learn to become good problem solvers, they’ll be able to solve all sorts of problems— not merely the single (sometimes atypical) problem embodied in a particular performance test.

M08_POPH0936_10_SE_C08.indd 221M08_POPH0936_10_SE_C08.indd 221 09/11/23 6:21 PM09/11/23 6:21 PM

222 Chapter 8 performance assessment

So, to provide helpful instructional illumination—that is, to assist the teacher’s instructional decision making—a rubric’s evaluative criteria dare not be task spe- cific. Instead, those criteria must be rooted in the skill itself. This can be illustrated by your considering an evaluative criterion frequently employed in the rubrics used to judge students’ written communication skills—namely, the evaluative criterion of organization. Presented here is a task-specific evaluative criterion that might be used in a scoring rubric to judge the quality of a student’s response to the following task:

“Compose a brief narrative essay, of 400 to 600 words, describing what happened during yesterday’s class session when we were visited by the local firefighters who described a fire-exit escape plan for the home.”

A task-specific evaluative criterion for judging the organization of stu- dents’ narrative essays: Superior essays will (1) commence with a recounting of the particular rationale for home fire-escape plans the local firefighters pro- vided, then (2) follow up with a description of the six elements in home-safety plans in the order in which those elements were presented, and (3) conclude by citing at least three of the life–death safety statistics the firefighters provided at the close of their visit. Departures from these three organizational elements will result in lower evaluations of essays.

Suppose you’re a teacher who has generated or been given this task-specific evaluative criterion. How would you organize your instruction? If you really tried to gain instructional guidance from this evaluative criterion, you’d be aiming your instruction directly at a particular task—in this case, a narrative account of the firefighters’ visit to your class. You’d need to teach your students to commence their narrative essays with a rationale when, in fact, using a rationale for an intro- duction might be inappropriate for other sorts of narrative essays. Task-specific evaluative criteria do not help teachers plan instructional sequences that promote their students’ abilities to generalize the skills they acquire. Task-specific criteria are just what they are touted to be—specific to one task.

During recent years, educators have seen a number of rubrics that classroom teachers have been required to generate on their own. From an instructional per- spective, many of these rubrics don’t pass muster. Numerous of these rubrics are simply swimming in a sea of task specificity. Such task-specific rubrics may make it easier to score students’ constructed responses. Thus, for scoring purposes, the more specific evaluative rubrics are, the better. But task-specific rubrics do not provide teachers with the kinds of instructional insights that good rubrics should. That is, they fail to provide sufficient clarity that allows a teacher to organize and deliver weakness-focused, on-target instruction.

You’ll not be surprised to learn that when the nation’s large testing firms set out to score thousands of students’ responses to performance tasks, they almost always employ task-specific rubrics. Such task-focused scoring keeps down costs. But regrettably, the results of such scoring typically supply teachers with few instructional insights.

M08_POPH0936_10_SE_C08.indd 222M08_POPH0936_10_SE_C08.indd 222 09/11/23 6:21 PM09/11/23 6:21 PM

rubrics: tWe ­retcWed and tWe rapturous 223

Hypergeneral Rubrics Another variety of scoring rubric that will not help teachers plan their instruction is referred to as a hypergeneral rubric. Such a rubric is one in which the evaluative criteria are described in exceedingly general and amorphous terms. The evalu- ative criteria are so loosely described that, in fact, teachers’ instructional plans really aren’t benefited. For example, using the previous example of a task calling for students to write a narrative essay, a hypergeneral evaluative criterion for organization might resemble the following one:

A hypergeneral evaluative criterion for judging the organization of students’ narrative essays: Superior essays are those in which the essay’s content has been arranged in a genuinely excellent manner, whereas inferior essays are those displaying altogether inadequate organization. An adequate essay is one repre- senting a lower organizational quality than a superior essay, but a higher organi- zational quality than an inferior essay.

You may think that you’re being put on when you read the illustrative hyper- general evaluative criterion just given. But you’re not. Most seasoned teachers have seen many such hypergeneral scoring rubrics. These rubrics ostensibly clarify how teachers can score students’ constructed responses, but really these vague rubrics do little more than loosely redefine such quality descriptors as superior, proficient, and inadequate. Hypergeneral rubrics often attempt to draw distinctions among students’ performances no more clearly than among students’ performances in earning grades of A through F. Hypergeneral rubrics provide teachers with no genuine benefits for their instructional planning, because such rubrics do not give the teacher meaningfully clarified descriptions of the criteria to be used in evaluating the quality of students’ performances—and, of course, in promoting students’ abilities to apply a rubric’s evaluative criteria.

Robert Marzano, one of our field’s most respected analysts of educational research, has taken a stance regarding scoring guides that seems to endorse the virtues of the kinds of hypergeneral rubrics currently being reviled here. Marzano (2008) has urged educators to develop rubrics for each curricular aim at each grade level using a generic rubric that can be “applied to all content areas” (pp. 10–11). However, for a rubric to be applicable to all content areas, it must obviously be so general and imprecise that much of its instructional meaning is likely to have been leached from the rubric.

The midpoint (anchor) of this sort of hypergeneral rubric calls for no major errors or omissions on the student’s part regarding the simple or complex con- tent and/or the procedures taught. If errors or omissions are present in either the content or the procedures taught, these deficits lead to lower evaluations of the student’s performance. If the student displays “in-depth inferences and applica- tions” that go beyond what was taught, then higher evaluations of the student’s performance are given. Loosely translated, these generic rubrics tell teachers that a student’s “no-mistakes” performance is okay, a “some-mistakes” performance is less acceptable, and a “better-than-no-mistakes” performance is really good.

M08_POPH0936_10_SE_C08.indd 223M08_POPH0936_10_SE_C08.indd 223 09/11/23 6:21 PM09/11/23 6:21 PM

224 Chapter 8 performance assessment

When these very general rubrics are applied to the appraisal of students’ mas- tery of given curricular goals, such rubrics’ level of generality offers scant instruc- tional clarity to teachers about what’s important and what’s not. I have enormous admiration for Bob Marzano’s contributions through the years, and I count him as a friend. But in this instance, I fear my friend is pushing for a sort of far-too-general scoring guide that won’t help teachers do a better instructional job.

Skill-Focused Rubrics Well, because we have now denigrated task-specific rubrics and sneered at hypergeneral rubrics, it’s only fair to wheel out the type of rubric that helps score students’ responses, yet can also illuminate a teacher’s instructional plan- ning. Such scoring guides can be described as skill-focused rubrics because they really are conceptualized around the skill that is (1) being measured by the constructed-response assessment and (2) being pursued instructionally by the teacher. Based on the views of numerous teachers who have used a variety of scor- ing rubrics, many teachers report that creating a skill-focused scoring rubric prior to their instructional planning nearly always helps them devise a more potent instructional sequence.

For an example of an evaluative criterion you’d encounter in a skill-focused rubric, look over the following illustrative criterion for evaluating the organiza- tion of a student’s narrative essay:

A skill-focused evaluative criterion for judging the organization of students’ narrative essays: Two aspects of organization will be employed in the appraisal of students’ narrative essays: overall structure and sequence. To earn maximum credit, an essay must embody an overall structure containing an in- troduction, a body, and a conclusion. The content of the body of the essay must be sequenced in a reasonable manner—for instance, in a chronological, logical, or order-of-importance sequence.

Now, when you consider this evaluative criterion from an instructional perspective, you’ll realize that a teacher could sensibly direct instruction by (1) familiarizing students with the two aspects of the criterion—that is, over- all structure and sequence; (2) helping students identify essays in which overall structure and sequence are, or are not, acceptable; and (3) supplying students with gobs of guided and independent practice in writing narrative essays that exhibit the desired overall structure and sequence. The better students become in employing this double-barreled criterion in their narrative essays, the better those students will be at responding to a task such as the illustrative one we just saw about the local firefighters’ visit to the classroom. This skill-focused evalua- tive criterion for organization, in other words, can generalize to a wide variety of narrative-writing tasks, not just an essay about visiting fire-folk.

And even if teachers do not create their own rubrics—for instance, if rubrics are developed by a school district’s curriculum or assessment specialists—those teachers who familiarize themselves with skill-focused rubrics in advance of

M08_POPH0936_10_SE_C08.indd 224M08_POPH0936_10_SE_C08.indd 224 09/11/23 6:21 PM09/11/23 6:21 PM

rubrics: tWe ­retcWed and tWe rapturous 225

instructional planning will usually plan better instruction than will teachers who aren’t familiar with a skill-focused rubric’s key features. Skill-focused rubrics make clear what a teacher should emphasize instructionally when the teacher attempts to promote students’ mastery of the skill being measured. Remember, although students’ acquisition of knowledge is an important curricular aspiration for teachers, assessment of students’ knowledge can obviously be accomplished by procedures other than the use of performance tests.

Let’s now identify five rules you are encouraged to follow if you’re creating your own skill-focused scoring rubric—a rubric you should generate before you plan your instruction.

Rule 1: Make sure the skill to be assessed is significant. It takes time and trouble to generate skill-focused rubrics. It also takes time and trouble to score students’ responses by using such rubrics. Make sure the skill being pro- moted instructionally, and scored via the rubric, is worth all this time and trouble. Skills that are scored with skill-focused rubrics should rep- resent demanding accomplishments by students, not trifling ones.

Teachers should not be ashamed to be assessing their students with only a handful of performance tests, for example. It makes much more sense to measure a modest number of truly powerful skills properly than to do a shoddy job in measuring a shopping-cart full of rather puny skills.

Rule 2: Make certain that all the rubric’s evaluative criteria can be addressed instruction- ally. This second rule calls for you always to “keep your instructional wits about you” when generating a rubric. Most important, you must scrutinize every potential evaluative criterion in a rubric to make sure you can actu- ally teach students to master it.

This rule doesn’t oblige you to adhere to any particular instructional approach. Regardless of whether you are wedded to the virtues of direct instruction, indirect instruction, constructivism, or any other instruction- al strategy, what you must be certain of is that students can be taught to employ appropriately every evaluative criterion used in the rubric.

Rule 3: Employ as few evaluative criteria as possible. Because people who start thinking seriously about a skill will often begin to recognize a host of nuances associated with the skill, they sometimes feel compelled to set forth a lengthy litany of evaluative criteria. But for instructional pur- poses, as for many other missions, less here is more groovy than more. Try to focus your instructional attention on three or four evaluative cri- teria; you’ll become overwhelmed if you foolishly try to promote stu- dents’ mastery of a dozen evaluative criteria.

Rule 4: Provide a succinct label for each evaluative criterion. The instructional yield of a skill-focused rubric can be increased simply by giving each evaluative criterion a brief explanatory label. For instance, suppose you were trying to improve your students’ oral presentation skills. You might employ a

M08_POPH0936_10_SE_C08.indd 225M08_POPH0936_10_SE_C08.indd 225 09/11/23 6:21 PM09/11/23 6:21 PM

226 Chapter 8 performance assessment

skill-focused rubric for oral communication containing four evaluative criteria—delivery, organization, content, and language. These one-word, easy-to-remember labels will help remind you and your students of what’s truly important in judging mastery of the skill being assessed.

Rule 5: Match the length of the rubric to your own tolerance for detail. During educators’ earliest experiences with scoring rubrics, those early evaluative schemes were typically employed for use with high-stakes statewide or districtwide assessments. Such detailed rubrics were often intended to constrain scorers so their scores would be in agreement. But, as years went by, most educa- tors discovered that many classroom teachers consider fairly long, rather detailed rubrics to be altogether off-putting. Whereas those teachers might be willing to create and/or use a one-page rubric, they regarded a six-page rubric as wretchedly repugnant. More recently, therefore, we encounter rec- ommendations for much shorter rubrics—rubrics that rarely exceed one or two pages. In the generation of useful rubrics, brevity wins.

Of course, not all teachers are put off by very detailed rubrics. Some teachers, in fact, find that abbreviated rubrics just don’t do an adequate job for them. Such detail-prone teachers would rather work with rubrics that specifically spell out just what’s involved when differentiating between quality levels regarding each evalua- tive criterion. A reasonable conclusion, then, is that rubrics should be built to match the level-of-detail preferences of the teachers involved. Teachers who believe in brevity should create brief rubrics, and teachers who believe in detail should create lengthier rubrics. If school districts (or publishers) are supplying rubrics to teachers, ideally both a short and a long version will be provided. Let teachers decide on the level of detail that best meshes with their own tolerance or need for detail.

So far, skill-focused rubrics have been touted in this chapter as the most use- ful for promoting students’ mastery of really high-level skills. Such advocacy has been present because of the considerable instructional contributions that can be made by skill-focused rubrics. But here’s a requisite caution. If you are employing a rubric that incorporates multiple evaluative criteria, beware of the ever-present temptation to sum those separate per-criterion scores to come up with one, big “overall” score. Only if the several evaluative criteria, when squished together, still make interpretive or instructional sense should such scrunching be undertaken.

But please do not feel obliged to carve out a skill-focused rubric every time you find yourself needing a rubric. There will be occasions when your instruction is not aimed at high-level cognitive skills, but you still want your students to achieve what’s being sought of them. To illustrate, you might want to determine your students’ mas- tery of a recently enacted federal law—a law that will have a significant impact on how American citizens elect their legislators. Well, suppose you decide to employ a constructed-response testing strategy for assessing your students’ understanding of this particular law. More specifically, you want your students to write an original, explana- tory essay describing how certain of the law’s components might substantially influ- ence U.S. elections. Because, to evaluate your students’ essays, you’ll need some sort

M08_POPH0936_10_SE_C08.indd 226M08_POPH0936_10_SE_C08.indd 226 09/11/23 6:21 PM09/11/23 6:21 PM

ratings and Observations 227

of scoring guide to assist you, what sort of rubric should you choose? Well, in that sort of situation, it is perfectly acceptable to employ a task-specific rubric rather than a skill- focused rubric. If there is no powerful, widely applicable cognitive skill being taught (and assessed), then the scoring can properly be focused on the given task involved, rather than on a more generally applicable skill.

Ratings and Observations Once you’ve selected your evaluative criteria, you need to apply them reliably to the judgment of students’ responses. If the nature of the performance-test task calls for students to create some sort of product, such as a written report of an experi- ment carried out in a biology class, then at your leisure you can rate the product’s quality in relation to the criteria you’ve identified as important. For example, if you had decided on three criteria to use in evaluating students’ reports of biology experiments, and could award from 0 to 4 points for each criterion, then you could leisurely assign from 0 to 12 points for each written report. The more clearly you understand what each evaluative criterion is, and what it means to award differ- ent numbers of points on whatever scale you’ve selected, the more accurate your scores will be. Performance tests that yield student products are definitely easier to rate, because you can rate students’ responses when you’re in the mood.

It is often the case with performance tests, however, that the student’s perfor- mance takes the form of some kind of behavior. With such performance tests, it will usually be necessary for you to observe the behavior as it takes place. To illustrate, suppose that you are an elementary school teacher whose fifth-grade students have been carrying out fairly elaborate social studies projects culminating in 15-minute oral reports to classmates. Unless you have the equipment to videotape your stu- dents’ oral presentations, you’ll have to observe the oral reports and make judg- ments about the quality of a student’s performance as it occurs. As was true when scores were given to student products, in making evaluative judgments about stu- dents’ behavior, you will apply whatever criteria you’ve chosen and assign what you consider to be the appropriate number of points on whatever scales you are using.

For some observations, you’ll find it sensible to make instant, on-the-spot quality judgments. For instance, if you are judging students’ social studies reports on the basis of (1) content, (2) organization, and (3) presentation, you might make observation-based judgments on each of those three criteria as soon as a report is finished. In other cases, your observations might incorporate a delayed evaluative approach. For instance, let’s say that you are working with students in a speech class on the elimination of “filler words and sounds,” two of the most prominent of which are starting a sentence with “Well” and interjecting frequent “uh”s into a presentation. In the nonevaluative phase of the observation, you could simply count the number of “well”s and “uh”s uttered by a student. Then, at a later time, you could decide on a point allocation for the criterion “avoids filler words and sounds.” Putting it another way, systematic observations may be set up so you

M08_POPH0936_10_SE_C08.indd 227M08_POPH0936_10_SE_C08.indd 227 09/11/23 6:21 PM09/11/23 6:21 PM

228 Chapter 8 performance assessment

make immediate or delayed allocations of points for the evaluative criteria you’ve chosen. If the evaluative criteria involve qualitative factors that must be appraised more judgmentally, then on-the-spot evaluations and point assignments are typi- cally the way to go. If the evaluative criteria involve more quantitative factors, then a “count now and judge later” approach usually works better.

Sources of Error in Scoring Student Performances When scoring student performances, there are three common sources of error that can contribute to inaccurate inferences. First, there is the scoring scale. Second, there are the scorers themselves, who may bring a number of bothersome biases to the enterprise. Finally, there are errors in the scoring procedure—that is, the process by which the scorers employ the scoring scale.

Scoring-Instrument Flaws The major defect with most scoring instruments is the lack of descriptive rigor with which the evaluative criteria to be used are described. Given this lack of rigor, ambiguity exists in the interpretations that scorers make about what the scoring criteria mean. This typically leads to a set of unreliable ratings. For exam- ple, if teachers are to rate students on the extent to which students are “control- ling,” some teachers may view this as a positive quality and some may view it as a negative quality. Clearly, an inadequately clarified scoring form can lead to all sorts of “noise” in the scores provided by teachers.

Procedural Flaws Among common problems with scoring students’ responses to performance tests, we usually encounter demands on teachers to rate too many qualities. Overwhelmed scorers are scorers rendered ineffectual. Teachers who opt for a large number of evaluative criteria are teachers who have made a decisively inept opt. Care should be taken that no more than three or four evaluative criteria are to be employed in evaluations of students’ responses to your performance assess- ments. Generally speaking, the fewer the evaluative criteria used, the better.

Teachers’ Personal-Bias Errors If you recall Chapter 5’s consideration of assessment bias, you’ll remember that bias is clearly an undesirable commodity. Teachers, albeit unintentionally, are frequently biased in the way they score students’ responses. Several kinds of personal-bias errors are often encountered when teachers score students’ con- structed responses. The first of these, known as generosity error, occurs when a

M08_POPH0936_10_SE_C08.indd 228M08_POPH0936_10_SE_C08.indd 228 09/11/23 6:21 PM09/11/23 6:21 PM

Sources of error in Scoring Student performances 229

teacher’s bias leads to higher ratings than are warranted. Teachers with a procliv- ity toward generosity errors see good even where no good exists.

At the other extreme, some teachers display severity errors. A severity error, of course, is a tendency to underrate the quality of a student’s work. When a stu- dent’s product deserves a “good,” teachers suffering from this personal-bias error will award the product only an “average” or even a “below average.”

Another sort of personal-bias error is known as central-tendency error. This describes a tendency for teachers to view everything as being “in the middle of the scale.” Very high or very low ratings are assiduously avoided by people who exhibit central-tendency error. They prefer the warm fuzziness of the mean or the median. They tend to regard midpoint ratings as inoffensive—hence they dispense midpoint ratings perhaps thoughtlessly and even gleefully.

A particularly frequent error arises when a teacher’s overall impression of a student influences how the teacher rates that student with respect to an individual criterion. This error is known as the halo effect. If a teacher has a favorable attitude toward a student, that student will often receive a host of positive ratings (deserved or not) on a number of individual criteria. Similarly, if a teacher has an unfavorable attitude toward a student, the student will receive a pile of negative ratings on all sorts of separate criteria. One nifty way to dodge halo effect, if the teacher can pull it off, is to score responses anonymously whenever this is a practical possibility.

But ­Wat Does tWis have to Do witW teacWingt Teachers are busy people. (This perceptive insight will not come as a surprise to anyone who really knows what a teacher’s life is like.) But busy people who survive in their task-buffeted lives will typically try to manage their multiple responsibilities efficiently. Busy people who are teachers, therefore, need to make sure they don’t expend their finite reservoirs of energy unwisely. And that’s where caution needs to be exercised with respect to the use of performance tests. Performance testing, you see, can be so very seductive.

Because performance tests are typically focused on measuring a student’s mastery of real-world, significant skills, such tests are appealing. If you’re an English teacher, for example, wouldn’t you rather determine whether your students possess composition skills by having them whip out actual “written-from-scratch” compositions than by having them merely spot punctuation problems in multiple- choice items? Of course, you would.

But although the allure of performance assessment is considerable, you should never forget that it takes time! And the time-consumption requirements of performance testing can readily eat into a teacher’s daily 24-hour allotment. It takes time for teachers to come up with defensible tasks for performance tests, time to devise suitable rubrics for scoring students’ responses, and, thereafter, time to score those responses. Performance assessment takes time. But, of course, proper performance testing not only helps teachers aim their instruction in the right directions; it also allows students to recognize the nature of the skill(s) being promoted.

So, the trick is to select judiciously the cognitive skills you will measure via performance tests. Focus on a small number of truly significant skills. You’ll be more likely to maintain your instructional sanity if you employ only a handful of high-level performance tests. Too many performance tests might push a teacher over the edge.

M08_POPH0936_10_SE_C08.indd 229M08_POPH0936_10_SE_C08.indd 229 09/11/23 6:21 PM09/11/23 6:21 PM

230 Chapter 8 performance assessment

One way to minimize halo effect, at least a bit, is occasionally to reverse the order of the high and low positions on the scoring scale so the teacher cannot unthinkingly toss out a whole string of positives (or negatives). What you really need to do to avoid halo effect when you’re scoring students’ responses is to remember it’s always lurking in the wings. Try to score a student’s responses on each evaluative criterion by using that specific criterion, not a contaminated general impression of the student’s ability.

Thinking back to Chapter 5 and its treatment of educational fairness, please recognize that in the scoring of students’ responses to any sort of student-generated product or behavior, teachers are typically tromping around in terrain that’s just brimming with the potential for unfairness. These are instances in which teachers need to be especially on guard against unconsciously bring- ing their personal biases into the scoring of students’ responses. Fairness is one of educational testing’s big three requisites (along with reliability and validity). When evaluating students’ performances, unfairness can frequently flower. One of the most straightforward ways of spotting fairness-flaws in your teacher- made tests is to mentally plop yourself into the test-taking seats of students who, because they are different (according to the subgroup label they carry), might be offended or unfairly penalized by what’s in your test. If you spot some potential problems, then do something that you think has a reasonable chance of fixing the problem. In short, put your unfairness-detection machinery into overdrive, and then follow up this unfairness-detection effort with your best effort to fix it.

What Do Classroom Teachers Really Need to Know About Performance Assessment? Performance assessment has been around for a long, long while. Yet, in recent years, a growing number of educators have become strong supporters of this form of assessment because it (1) represents an alternative to traditional paper-and-pencil tests and (2) is often more authentic—that is, more reflective of tasks people routinely need to perform in the real world. One of the things you need to understand about performance assessment is that it differs from more conventional assessment chiefly in the degree to which the assessment task matches the skills and/or knowledge about which you wish to make inferences. Because performance tasks often coincide more closely with high-level cognitive skills than do paper-and-pencil tests, more accurate interpretations can often be derived about students. Another big plus for performance tests is that they estab- lish assessment targets which, because such targets often influence the teacher’s instruction, can have a positive impact on instructional activities.

The chapter’s final admonitions regarding the biases that teachers bring to the scoring of students’ performance-test responses should serve as a reminder.

M08_POPH0936_10_SE_C08.indd 230M08_POPH0936_10_SE_C08.indd 230 09/11/23 6:21 PM09/11/23 6:21 PM

CWapter Summary 231

Chapter Summary Although this chapter dealt specifically with performance assessment, a number of the points made in the chapter apply with equal force to the scoring of any type of constructed-response items such as those used in essay tests or short-answer tests. After defining performance tests as a mea- surement procedure in which students create original responses to an assessment task, it was pointed out that performance tests differ from more conventional tests primarily in the degree to which the test situation approximates the real- life situations to which inferences are made.

The identification of suitable tasks for per- formance assessments was given considerable

attention in the chapter because unsuitable tasks will surely lead to unsatisfactory performance assessments. Seven evaluative criteria were supplied for performance-test tasks: (1) gener- alizability, (2) authenticity, (3) multiple foci, (4) teachability, (5) fairness, (6) feasibility, and (7) scorability. Particular emphasis was given to selecting tasks about which defensible inferences could be drawn regarding students’ generalized abilities to perform comparable tasks.

The significance of the skill to be assessed via a performance task was stressed. Next, evaluative criteria were defined as the factors by which the acceptability of a student’s performance is judged.

parent talk The vice president of your school’s Parent Advisory Council has asked you to tell him why so many of your school’s teachers are now assessing students with performance tests instead of the more traditional paper-and-pencil tests.

If I were you, here’s how I’d respond:

“The reason why most of us are increasingly relying on performance tests these days is that performance tests almost always measure higher-level student skills. With traditional tests, too many of our teachers found that they were inadvertently assessing only the students’ abilities to memorize facts.

“In recent years, educators have become far more capable of devising really demanding performance tests that require students to display genuinely high-level intellectual skills—skills that are way, way beyond memorization. We’ve learned how to develop the tests and how to score students’ responses using carefully developed scoring rules that we refer to as rubrics.

“Please stop by my classroom and I’ll show you some examples of these demanding performance tests. We’re asking more from our students, and— we have evidence galore—that we’re getting it.”

Now, how would you respond to the vice president of the Parent Advisory Council?

If you employ performance tests frequently in your classrooms, you’ll need to be careful at every step of the process—from the original conception and birthing of a performance test down to the bias-free scoring of students’ responses. This is not fool’s play for sure. But, given the instructional targets on which many perfor- mance tests are based, it can definitely be worth the effort. A first-rate educational program can be rooted in a top-drawer assessment program—and this is often a top-drawer performance-test assessment program.

M08_POPH0936_10_SE_C08.indd 231M08_POPH0936_10_SE_C08.indd 231 09/11/23 6:21 PM09/11/23 6:21 PM

232 Chapter 8 performance assessment

The evaluative criteria constitute the most impor- tant features of a rubric that’s employed to score student responses. The significance of selecting suitable evaluative criteria was emphasized. Once the criteria have been identified, a numerical scor- ing scale, usually consisting of from 3 to 6 score points, is devised for each evaluative criterion. The evaluative criteria are applied to student

performances in the form of ratings (for student products) or observations (for student behaviors).

Distinctions were drawn among task-specific, hypergeneral, and skill-focused rubrics. Deficits in the former two types of rubrics render them less appropriate for supporting a teacher’s classroom instructional effort. A skill-focused rubric, how- ever, can markedly enhance a teacher’s instruction.

References Brookhart, S. (2013). How to create and use rubrics

for formative assessment and grading. ASCD. Brookhart, S. (2023). Classroom assessment

essentials. ASCD. Chowdhury, F. (2019). Application of rubrics in

the classroom: A vital tool for improvement in assessment, feedback and learning, International Education Studies, 12(1): 61–68. https://files.eric. ed.gov/fulltext/EJ12 01525.pdf

Education Week. (2019, February 25). K–12 Performance assessment terms explained [Video]. Retrieved September 20, 2022, from https://www.edweek.org/teaching- learning/ video-k-12-performance- assessment-terms-explained/2019/02

Fitzpatrick, R., & Morrison, E. J. (1971). Performance and product evaluation. In E. L. Thorndike (Ed.), Educational measurement (pp. 237–270). American Council on Education.

Lane, S. (2013). Performance assessment. In J. H. McMillan (Ed.), SAGE Handbook of research on classroom assessment (pp. 313–329). SAGE Publications.

Marzano, R. J. (2008). Vision document. Marzano Research Laboratory.

McTighe, J., Doubet, K. J., & Carbaugh, E. M. (2020). Designing authentic performance tasks and projects: Tools for meaningful learning and assessment. ASCD.

Sawchuk, S. (2019, February 6). Performance assessment: 4 Best practices, Education Week. Retrieved September 20, 2022, from https://www.edweek.org/teaching- learning/performance-assessment-4-best- practices/2019/02

Wormeli, R. (2018). Fair isn’t always equal: Assessment & grading in the differentiated classroom (2nd ed.). Stenhouse Publishers.

M08_POPH0936_10_SE_C08.indd 232M08_POPH0936_10_SE_C08.indd 232 09/11/23 6:21 PM09/11/23 6:21 PM

a testing takeaway 233

A Testing Takeaway

Performance Tests: Tension in the Assessment World* W. James Popham, University of California, Los Angeles

Educators test their students to determine what those students know and can do. This simplified depiction of educational assessment captures the rationale for teachers’ testing of their students. Yet, as is the case with most simplifications, often lurking within such no-frills explanations are issues of considerable complexity. In educational testing, one of these rarely discussed complexities is the appropriate use of performance testing. Let’s briefly consider it.

Even though all educational tests require students to “perform” in some way, most conventional test items ask students to respond (on paper or computer) to stimuli presented by others. In contrast, the essential element of a performance test is that the test-taker is required to construct an original response—either as a process that can be evaluated as it takes place (for example, an impromptu speech) or as a product (for instance, a bookend made in woodshop) that’s subsequently evaluated.

A performance test typically simulates the criterion situation; that is, it approximates the real-world setting to which students must apply a skill. More conventional assessment, such as paper-and-pencil tests and computer-administered exams, are clearly limited in the degree to which they can approximate lifelike demands for students’ skills. Let’s turn, however, to an emerging conflict about performance testing.

The most prominent reason for advocating more widespread performance testing is that whatever is tested in our schools will almost certainly end up being taught in our classrooms. If we can more frequently measure students’ mastery of a wide range of real-world skills, such as composing persuasive essays or organizing a monthly budget, then more skills related to such performance tests will surely be taught.

During recent decades, educators were sometimes asked to increase their performance testing, but currently such demands seem to have slackened. This arises because whatever the instructional dividends of performance tests, they are expensive to score. The more elaborate the performance test, the more expensive it is to train and supervise scorers. Even though computers can be “trained” to provide accurate scores for uncomplicated performance tasks, for the foreseeable future we must often rely on costly human scorers.

Hence, the positives and negatives of such assessments:

• Pro: Performance tests can simulate real-world demands for school-taught skills.

• Con: Employing performance tests is both costly and time-consuming.

Because better performance testing leads to better taught students, do what you can to support funding for the wider use of performance tests. It’s an educationally sensible investment.

*From CWapter 8 of Classroom Assessment: What Teachers Need to Know, 10tW ed., by ­. James popWam. CopyrigWt 2022 by pearson, wWicW Wereby grants permission for tWe reproduction and distribution of tWis Testing Takeaway, witW proper attribution, for tWe intent of increasing assessment literacy. a digitally sWareable version is available from Wttps://www.pearson.com/store/en-us/pearsonplus/login.

M08_POPH0936_10_SE_C08.indd 233M08_POPH0936_10_SE_C08.indd 233 09/11/23 6:21 PM09/11/23 6:21 PM