PLAGIARISM FREE "A" WORK

NeNe1994

Chapter3TooManyTestingTargets.pdf

Home >Education homework help >PLAGIARISM FREE "A" WORK

2 9

TEACHERS WANT TO FIND OUT WHAT’S GOING ON INSIDE STUDENTS’ HEADS. PUT-

ting it a bit more formally, teachers want to make inferences about

the nature of their students’ skills, knowledge, and affect. And that’s

the precise point at which the integral relationship between curricu-

lum and assessment takes center stage. You see, testing doesn’t take

place in a vacuum. More specifically, testing doesn’t take place in a

curriculum-free vacuum. Typically, teachers test students to get a fix

on their students’ status with respect to those cognitive or affective

targets that have been laid out in a prescribed curriculum. More often

than not, therefore, teachers find themselves trying to “test what’s in

the curriculum.”

As a result, most educators today also end up trying to solve an

essentially insolvable problem: how to satisfactorily assess students’

mastery of too many curricular aims imposed “from above.” There’s

an oft-uttered saying that you can’t make a silk purse out of a sow’s

ear. The gist of the saying’s message, of course, is that if you are given

unsuitable raw material, it’s impossible to transform that raw material

into something spectacular. When teachers are informed that they

should promote their students’ mastery of a huge number of state-

stipulated or district-stipulated curricular goals, many teachers actu-

ally try to do so. Unfortunately, that’s almost always a mistake.

3 Too Many

Testing Targets

ch3.qxd 7/10/2003 9:55 AM Page 29

Popham, W. James. Test Better, Teach Better : The Instructional Role of Assessment, Association for Supervision & Curriculum Development, 2003. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/amridge/detail.action?docID=5704436. Created from amridge on 2022-01-13 03:37:00.

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

T E S T B E T T E R , T E A C H B E T T E R3 0

Assessment and curriculum are not a pair of isolated enclaves,

each functioning happily without any reference to the other.

Teachers ought to test what students ought to learn and, yes, what’s

set forth in a curriculum should be the dominant determiner of the

sorts of classroom tests that teachers give to their students. But too

many educators reckon that assessment programs can blossom irre-

spective of the curricula from which they spring. State-level testing

does not occur apart from the test-developers’ attention to state cur-

ricula. And, just as surely, classroom-level testing should not occur

apart from a teacher’s attention to curricular considerations. Thus, if

teachers are dealing with a sow’s-ear curriculum, they’ll never be able

to create silk-purse assessments. It’s important, therefore, to know

what you can do if you find yourself forced to cope with a curriculum

prescription that contains too many pills in its bottle. In this chapter,

I’ll look at the origins of the problem and then offer my recommen-

dations on how to solve it in your own classroom.

Standards-Based Reform A half-century ago, almost every U.S. state set out an officially

approved curriculum for its schools. This authorized curriculum,

sometimes called a “state curriculum syllabus,” identified the knowl-

edge and skills that the state’s teachers were supposed to promote at

different grade levels (or grade ranges) and in most subjects. Frankly,

few educators paid much attention to those curricular documents.

State curricular syllabi nestled quietly in teachers’ desk drawers, rarely

to be opened.

In the 1990s, many educational policymakers began to proclaim

the virtues of standards-based reform as a new strategy to reform pub-

lic schools. Here’s how it was intended to work. First, a group of expe-

rienced teachers and curriculum specialists in each state met to stake

out a set of challenging content standards for each grade level, or for

grade ranges, in all key subjects. (In most instances, these collections

of content standards were dead ringers for yesteryear’s curricular

ch3.qxd 7/10/2003 9:55 AM Page 30

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

T o o M a n y T e s t i n g T a r g e t s 3 1

syllabi.) Then, a statewide test was either selected or constructed to

assess students’ mastery of those content standards. Finally, students’

test performances were employed to identify the schools (and teach-

ers) whose students were achieving high scores or low scores. High-

scoring schools were regarded as winners; low-scoring schools were

regarded as losers.

The underlying strategy of standards-based reforms rested solidly

on the proposition that by administering standards-based tests annu-

ally to students at selected grade levels, the state’s teachers would be

spurred to effectively promote students’ content-standard mastery (as

reflected by higher test scores). Those higher test scores would, of

course, signify improved student mastery of the state’s challenging

content standards.

Unfortunately, there turned out to be several significant short-

comings in the way that the educational officials in most states tried

to implement standards-based reform. For one thing, typically, the

quality of the alignment between state-approved content standards

and state-approved achievement tests has been very weak. Rather

than devote scarce educational funds to the development of cus-

tomized tests based exclusively on a state’s curriculum, many states

simply chose to use an off-the-shelf standardized achievement test.

Such tests were selected despite the fact that many of the state’s

approved content standards appeared to be addressed by only one or

two items in those off-the-shelf tests. And, worse, in some instances

there were state content standards that weren’t measured by even one

item in those already-built standardized achievement tests.

Rarely, however, do even custom-built statewide standards-based

tests include sufficient numbers of items to provide evidence of a stu-

dent’s status with respect to the mastery of particular content stan-

dards. That’s a serious shortcoming, because unless teachers, students,

and parents can determine which content standards are being mas-

tered and which ones are not, then standards-based reform is doomed

to be little more than an attractive but meaningless bit of rhetoric.

ch3.qxd 7/10/2003 9:55 AM Page 31

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

T E S T B E T T E R , T E A C H B E T T E R3 2

You see, if we don’t know which content standards have been suc-

cessfully promoted, then teachers can’t tell whether particular parts of

their instructional programs have been effective or ineffective. But

there’s an equally serious deficiency with most of today’s standards-

based reform efforts, and that problem lies in the curricular corner-

stones of this entire reform strategy: the content standards themselves.

Typically, the people who carve out the content standards for any

given state are a group of experienced educators, perhaps 20–30 such

individuals, who have been selected for the task because of their

expertise in the subject areas for which the content standards are

being identified. For instance, the Content Standards Committee of

25 educators in State X who have been directed to identify the knowl-

edge and skills in science that State X students in grades 4–8 should

master is typically composed of science-education specialists.

Specialists, as we all know, tend to love their specialties. That’s the

strength and the weakness of the way that most states have identified

their content standards. When subject-matter specialists are asked to

isolate what students need to know about their specialty, the special-

ists’ response will almost always be “Everything!” In other words,

because the people who take part in the identification of content

standards typically revere their subject fields, they often identify all

the knowledge and all the skills that they wish well-taught children

at a given age would possess.

The net effect of this kind of wish-list curricular thinking is that,

in most states, the number of content standards officially approved

for most subjects is enormous. These wish-list content standards sim-

ply aren’t teachable in the instructional time available to the class-

room teachers. It’s equally unrealistic to think that these litanies of

content standards can be satisfactorily assessed by an annual state-

wide test of 50–60 items per subject area. In most instances, only a

small proportion of these lengthy lists of state-approved content

standards are measured at all, even by one puny item.

ch3.qxd 7/10/2003 9:55 AM Page 32

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

T o o M a n y T e s t i n g T a r g e t s 3 3

Think of the teachers in states where wish-list content standards

abound. You can be certain that there is nothing even approximating

total measurement of these standards. Unquestionably, therefore,

these teachers will face some absurd choices. The educational-reform

strategy in place has likely attracted substantial political support in

high places. (Politicians typically revel in rallying behind the “pro-

motion of high standards.”) Moreover, most citizens joyfully applaud

the merits of any accountability strategy aimed at promoting stu-

dents’ mastery of “demanding” content standards. And yet, the two

most important components of this accountability strategy typically

malfunction. The content standards are far too numerous to teach,

and far, far too numerous to test. The culminating high-stakes tests

used to assess students’ mastery of those standards don’t actually

measure those standards and, worse, don’t help teachers make defen-

sible instructional decisions.

Coping with Unrealistic Curricular Demands Teachers who are being asked to promote their students’ mastery of

an unwieldy array of state-sanctioned content standards, and to do so

without receiving test results that reveal students’ standard-by-

standard status, are in an untenable instructional situation. But even

though such expectations may be unrealistic, those expectations are

still very real. And there are adverse consequences for teachers who

fall short. How, then, should teachers cope?

My advice is to select a coping strategy, given your available alter-

natives, that is most likely to benefit your students. One popular but

unwise response to a rambling, excessive set of standards-based

expectations—an aiming-for-a-silk-purse response that most certainly

won’t help students—is to try to “cover” all the state-stipulated stan-

dards. Shallow coverage, almost by definition, will rarely lead to deep

understanding, especially when teachers must race on to tomorrow’s

content standards before students have the chance to master today’s.

ch3.qxd 7/10/2003 9:55 AM Page 33

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

T E S T B E T T E R , T E A C H B E T T E R3 4

It would be far better, of course, if a state’s policymakers would

realize that they’ve given teachers too much to teach (and test) in the

time available, and, after some serious curricular soul-searching, they

would attach estimates of importance to their too-long sets of con-

tent standards. Thereafter, the state’s teachers could determine which

curricular outcomes were really the most imperative to foster. What’s

more, a more modest number of super-significant content standards

would make it possible to create standards-based tests that could

report students’ results on a standard-by-standard basis, thus provid-

ing the explicit standards-mastery information that teachers could

use to refine or revise their instructional approaches.

However, many teachers will find that their own state’s standards-

based reform strategy has not been improved along these lines. So,

given this sort of unhappy situation, my best advice to teachers is to

prioritize for yourself. Either on your own or (even better) in collegial

teams, review the complete set of content standards for which you

are responsible, and set your own priorities.

Here’s how I think the prioritizing game should be played. First,

consider every single content standard on the state- or district-

approved list for the subject area or areas in which you teach. For each

content standard in your subject, assign one of three ratings:

• Essential. It is absolutely necessary for my students to have mas-

tered this content standard by the end of my instruction.

• Highly desirable. It very important for my students to have mas-

tered this content standard by the end of my instruction.

• Desirable. If possible, I would like my students to have mastered

this content standard by the end of my instruction.

After you’ve rated all the content standards, then you need to rank

only the ones that you already rated as “essential.” Rank all of those

essential content standards from most important (#1 on your list) to

next most important (#2), and so on. Then, design your instruction

ch3.qxd 7/10/2003 9:55 AM Page 34

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

T o o M a n y T e s t i n g T a r g e t s 3 5

so that you can do a thorough and effective job of promoting your

students’ mastery of as many of the highest prioritized content stan-

dards as you can promote in the instructional time you have avail-

able. Naturally, you’d like to have your students learn as many good

things as they possibly can, so you’ll also want to give at least some

instructional attention to those content standards you regarded as

“highly desirable” and “desirable.” But any such instructional atten-

tion need not be as extensive as the instructional attention you

devote to your highest-priority essential content standards.

Remember, what you’re trying to do is to get your students to master

the knowledge and skills that, in your professional judgment, are the

“very most important.”

And what about assessment? Well, although we’ve seen that total

measurement of innumerable content standards is impossible, espe-

cially if one is trying to get a fix on students’ per-standard mastery, it

is definitely possible to assess a more limited number of high-priority

content standards. And that’s just what you need to do.

In the last chapter, I pointed out how a somewhat vague content

standard can be operationalized by the assessment (or, preferably, by

the multiple assessments) employed to measure mastery of that vague

content standard. Consistent with that advice, what you need to do

is come up with suitable assessments for each of the high-priority con-

tent standards you’ve decided to tackle instructionally. The assess-

ments you choose will typically lead you to better instructional deci-

sions because of the increased clarity that those “operationalizing”

assessments will provide. Moreover, if you use pre-assessments to

determine what your students already know and can do, you’ll be

better able to choose suitable instructional activities for those stu-

dents. At the end of your instruction, of course, you’ll be able to get

a test-based estimate of how well your students have mastered each

high-priority content standard that you’ve set out to pursue.

Additionally, if you use any sort of less-formal en route assessments,

perhaps you’ll find that your students have mastered some content

ch3.qxd 7/10/2003 9:55 AM Page 35

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

T E S T B E T T E R , T E A C H B E T T E R3 6

standards more rapidly than you’d anticipated, thus enabling you to

move on to your next high-priority standard and, ultimately, give

instructional attention to other important standards that did not

make your initial cut.

Counteracting the Myth of Total Measurement It is impossible, in real schools, to assess students’ attainment of all

the lovely curricular aims we’d like those students to accomplish.

“Total measurement” is a myth. Moreover, the pretense that a

lengthy list of content standards can somehow be accurately meas-

ured breeds a corollary myth of “total instruction.” It is just as silly to

think that teachers can effectively promote students’ mastery of

seemingly endless content standards as it is to think that all those

content standards can be properly assessed.

Prioritizing offers a reasonable way out of this instructional

dilemma. (To my mind, it is the only way out of this bind.) Teachers

need to prioritize a set of content standards so they can identify the

content standards at which they will devote powerful, thoroughgoing

instruction, and then they need to formally and systematically assess

students’ mastery of only those high-priority content standards. Time

permitting, instructional attention to other, lower-priority content

standards could certainly be worked in. And other, less systematic

assessments could also be used to gauge students’ mastery of those

same lower-priority content standards.

Some might object to the prioritization strategy I have recom-

mended here because it seems to suggest that a teacher should be

doing less with students—should be focusing on fewer curricular aims.

To such criticism, I reply that there’s a whopping big difference

between content standards that are simply sought and content stan-

dards that are truly taught. Indeed, when teachers unthinkingly set

out to seek student mastery of so many curricular outcomes, what is

sought will, in fact, rarely be taught. The more ground that teachers

try to cover, the less likely it is they’ll be able to help students gain

ch3.qxd 7/10/2003 9:55 AM Page 36

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

T o o M a n y T e s t i n g T a r g e t s 3 7

deep understandings and facility with complex skills. That’s what

“challenging” content standards call for, and that’s what everyone

really wants.

I’m suggesting that teachers, and the administrators they report

to, need to be honest about what’s instructionally accomplishable in

a finite amount of teaching time. And that kind of honesty leads me

to encourage you to do a serious prioritizing job on any curricular

aims that have been officially tossed into your lap. If the pressure to

“cover all content standards” is really severe, then you may wish to

give really rapid-fire treatment to the whole set of content standards,

but give your genuine instructional and assessment attention to only

the curricular aims you believe to be truly of the highest significance.

My proposed prioritization strategy was initially spawned in

response to ill-conceived, state reform strategies built around exces-

sively lengthy lists of content standards. Please recognize, however,

that this prioritization strategy can work just as effectively when

teachers are trying to figure out how best to organize their own

instruction and to evaluate its effectiveness. After all, even if a teacher

had no official state-imposed content standards to wrestle with, that

teacher would still have to decide what to emphasize instructionally

and what to emphasize from an assessment perspective. There’s lots

of stuff to teach, and there’s lots of stuff to test. Teachers will do a bet-

ter job for their students if they devote serious effort to identifying

the most significant things that can be taught in the time available.

Then those high-priority curricular aims can be well taught and well

tested. A prioritization strategy is clearly predicated on a less-is-more

approach to teaching and testing. And it works!

A Labyrinth of Labels Now I want to change my focus for just a bit. In this chapter, I’ve tried

to tackle a curricular problem that is causing today’s teachers a heap

of instructional and assessment headaches. The essence of my mes-

sage is that excessive curricular aspirations can make instructionally

ch3.qxd 7/10/2003 9:55 AM Page 37

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

sensible assessment impossible. Thus, when teachers are tussling with

their own testing issues, it’s rare that curriculum considerations aren’t

involved.

And because that’s so, I want to close out the chapter with a brief

treatment of several curriculum-related labels that today’s classroom

teachers are almost sure to encounter. As you’ll see, teachers who are

baffled by the meaning of these labels will rarely be able to make the

best curricular or assessment decisions.

In the previous chapter, we saw that there are lots of ways to

describe curricular aims: goals, objectives, expectancies, benchmarks,

learning outcomes, and so on. Today’s most popular label is content

standards. In the No Child Left Behind Act, curricular aims are described

as academic content standards. Typically, classroom teachers pursue con-

tent standards stipulated by state or district authorities, although many

teachers sometimes add on a few noncompulsory content standards

that they personally regard as worthwhile.

There is one strong possibility of semantic confusion these days,

however, and that stems from the use of another descriptive label:

performance standards.* A performance standard refers to the required

level of proficiency students are expected to display when they have

mastered a content standard. For example, if a content standard calls

for a student to write a coherent paragraph, a performance standard

indicates how well that paragraph must be written. Or, if a content

standard focuses on the student’s acquisition of knowledge—say,

knowing the definitions of 50 biological terms—a performance stan-

dard describes how many of the 50 terms a student would need to

know in order to have “mastered” the content standard. Thirty-five

terms? Forty terms? Forty-five?

T E S T B E T T E R , T E A C H B E T T E R3 8

*The authors of the No Child Left Behind Act used the term academic achievement standards rather than “performance standards.” As the nation’s educators become more familiar with NCLB requirements, this phrase may become more widely used. However, for the time being, I’m going to stick with the more commonly employed terminology, namely, performance standards.

ch3.qxd 7/10/2003 9:55 AM Page 38

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

T o o M a n y T e s t i n g T a r g e t s 3 9

A content standard without an accompanying performance stan-

dard is like a play without a final act. You can’t tell what the content

standard really signifies until you know how students’ mastery of it

will be assessed and, thereafter, until you know what sort of per-

formance standard has been set on that assessment. Even the loftiest

content standard can be transformed into a trifling expectation if the

bar is set very low. Likewise, a trivial content standard can become a

challenging, praiseworthy expectation when it’s accompanied by a

super-stringent performance standard. For instance, if elementary

school students were to be able to name the capitals of all 50 states

(the content standard), you could set a mandatory 100 percent cor-

rect level (the performance standard) for proficient performance. The

result would be a tough task for students, even though the task deals

with a rather trivial curricular aim.

Increasingly, U.S. educators are building performance standards

along the lines of the descriptive categories used in the National

Assessment Educational Progress (NAEP), a test administered periodi-

cally under the auspices of the federal government. NAEP results per-

mit students’ performances in participating states to be compared, so

that it can be seen which states’ students outperform which other

states’ students. (Given Americans’ abiding preoccupation with win-

ners and losers, it’s surprising that no one has thought to report NAEP

results in newspapers’ sports sections.) At any rate, since 1990, NAEP

results have been described in four performance categories: advanced,

proficient, basic, and below basic. Most of the 50 states now use those

four categories or labels quite similar to them. For example, if stu-

dents were taking a statewide examination consisting of 65 multiple-

choice items, the performance standards for that test could be set by

deciding how many of the 65 items must be answered correctly for a

student to be classified as advanced, how many items for proficient,

and so on.

My point is that performance standards are malleable, and you

never know what something like “basic” means until you read the

ch3.qxd 7/10/2003 9:55 AM Page 39

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

T E S T B E T T E R , T E A C H B E T T E R4 0

fine-print description of that level of performance. For example, the

No Child Left Behind Act calls for states to establish at least three

levels of academic achievement standards (advanced, proficient, and

basic) and to demonstrate, over time, state-decreed increases in the

proportion of students deemed “proficient” or above. Because the

thrust of NCLB is to get all students to be proficient or advanced,

the legislation describes the basic level as “the progress of lower-

achieving children toward mastering the proficient and advanced

levels of achievement.” However, each state is allowed to define “pro-

ficient” in its own way. And because there are significant negative

sanctions for schools that fail to get enough students to score at the

proficient levels on NCLB tests, in some states there have been

remarkably lenient levels of “proficiency” established.

Clearly, because the two kinds of standards (content and per-

formance) have such distinctive purposes, teachers need to avoid the

confusion that’s likely to arise when one simply refers to “standards”

without a preceding adjective. I encourage you to avoid talking about

“high standards” or “challenging standards” because, as you’ve just

seen, if we really want our students to accomplish worthwhile things,

we’ll need high-level content standards accompanied by high-level

performance standards.

INSTRUCTIONALLY FOCUSED TESTING TIPS

• Learn which content standards are required by your state or

district.

• Determine if those content standards are accompanied by per-

formance standards.

• Prioritize any officially sanctioned content standards.

• Systematically assess only the high-priority content standards.

• Give major instructional attention to only the high-priority con-

tent standards.

ch3.qxd 7/10/2003 9:55 AM Page 40

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

T o o M a n y T e s t i n g T a r g e t s 4 1

Recommended Resources

Falk, B. (2000). The heart of the matter: Using standards and assessment to learn. Westport, CT: Heinemann.

Jacobs, H. H. (Program Consultant). (1991). Curriculum mapping: Charting the course for content [Videotape]. Alexandria, VA: Association for Supervision and Curriculum Development.

Kohn, A. (1999). The schools our children deserve: Moving beyond traditional classrooms and “tougher standards.” Port Chester, NY: National Professional Resources, Inc.

Kohn, A. (Program Consultant). (2000). Beyond the standards movement: Defending quality education in an age of test scores [Videotape]. Port Chester, NY: National Professional Resources, Inc.

Linn, R. L. (2000, March). Assessments and accountability. Educational Researcher, 29(2), 4–16.

ch3.qxd 7/10/2003 9:55 AM Page 41

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .