Discussion 5

Dimensions of Learning II

EDU 503

Belhaven University

Unit 5, Part 2

Assessment

1

Class Introduction

 Test-item development is a critical

component of classroom instruction. Test-

item development for day-to-day instructional

practice is often without blueprint specifics

and meeting the following targets:

 Measuring desired skill(s) and objective

 Minimum scoring time

 Objective rather than subjective

 Informing the daily instructional practice

2

Topics we’ll cover

 Developing Good Test Items—learn and explore

rules for creating a better test item for a valid

assessment.

3

Class Objectives

 Learn and explore rules for creating a better test

item for a valid assessment.

4

Biblical Foundation

 And whatever you do, in word or in deed, do

everything in the name of the Lord Jesus,

giving thanks to God the Father through him.

Colossians 3:17

5

Deciding on a testing format

6

 The choice of item format is sometimes

determined by your instructional objectives.

 At other times, the advantages and

disadvantages of the different formats should

influence your choice.

 What is the goal of your test?

 Design items where guessing is minimized

 Test items should be objective in scoring

Print the handout “Test Item Activity”; then record your answers on your printout

Pretest your knowledge: Determine whether each of the test items are good (G) or poor (P)

Interactive activity

True-False items

8

 True-false items

 require less time to construct

 are most prone to guessing  have a variety of other faults

 Faults include (features that make poor items):

 Absolutes in wording (i.e., always, all, never, only)

 Double negatives

 Opinionated

 Double-barreled statements (i.e., use of and, or)

 Excessive wordiness

 A tendency to reflect statements taken verbatim from readings (w/o context, or out of context)

Section I – True/False

T/F items are popular because they are easy and quick to write – or seem

to be; however, good T/F items are not that easy to write.

Instructions: Consider the following and use common sense to determine

which are good (G) and which are poor (P).

__1. High IQ children always get high grades in school

__2. Will Rogers stated, “I never met a man I didn’t like.”

__3. If a plane crashed on the Mexican-US border, half the survivors

would be buried in Mexico and half in the United States.

__4. The use of double negatives is not an altogether undesirable

characteristic of diplomats and academicians.

__5. Prayer should not be outlawed in schools.

__6. Of the objective items, true/false items are the least time consuming

to construct.

__7. The trend toward competency testing of high school graduates began

in the late 1970s and represents a big step forward for slow learners.

 1- always is an absolute and absolutes (all, always, never) cue student that the question is

false

 2-G – in order to answer, students would have to know that Will Rogers made the statement

 3-survivors are not buried, and students would suspect this is a trick question. This statement

should have perhaps used fatalities instead of survivors

 4-not and undesirable are both negatives – a single negative in a test question is confusing

enough for students, make positive statements instead – The use of double negatives is an

altogether desirable trait of diplomats and academicians. However, this item is still flawed in

that it states an opinion, not a fact. It is better written “According to the National Institute of

Diplomacy, the use of double negatives is a desirable trait of diplomats and academicians.”

 5-opinion – therefore not obviously true or false. Rewritten, “The ACLU has taken the position

that prayer should not be outlawed in schools” and Notice the negative not is italicized, when

using a negative, italicize!

 6-G-this is factual information from the text

 7-this is a double-barreled item…there are two parts, the learner has to determine if both are

true or both are false or is one false and one true … simply construct two items here

The trend toward competency testing of high school graduates began in the late 1970s

The trend toward competency testing represents a big step forward for slow learners

True/False key

1. Instructions for marking true or false should be explained

2. Construct statements that are definitely true or definitely false –

opinions must be attributed to a source

3. Use relatively short statements and eliminate extraneous material

4. Keep true and false statements at approximately the same length

and approximately of equal number

5. Avoid double negatives, absolutes and terms denoting indefinite

degrees (long time, regularly, never, only),

6. Avoid patterns TTFFTTFF OR TFTFTFTF and so on,

7. Avoid taking statements from the text and presenting them out of

context

8. And, always allowing the student the opportunity for a restricted

response to explain why the statement is false is good practice

How to construct good true/false items

Completion Items

Completion items (fill in the blank)  rival true-false items in ease of construction.  Since answers must be supplied, they are

least subject to guessing.  require more scoring time

Faults include:  Too many blanks  Lack of specificity (too many potential

responses)  Failure to state a problem

Section II - Completion Items

Instructions: Consider the following and use common sense to determine which

are good (G) and which are poor (P)

__1. The evolutionary theory of ______ is based on the principle of ________.

__2. Columbus discovered America in ______.

__3. The capital of Mexico is ______.

__4. In what year did William J. Clinton become president of the United States?

______

__5. ______ blanks cause much frustration in ______.

__6. ______ was the first American to ______.

Completion - Key

 1-avoid more than one blank per question. The evolutionary

theory of Darwin is based on the principle of survival of the

fittest

 2-most people would choose Good; however, a plausible

answer could be fifteenth century

 3-poor as “is larger than the capital of Alaska,” or “is a

beautiful city” … rewritten, “The name of the capital city of

Mexico is _____.”

 4-G because no other answer seems plausible here

 5 & 6 too many blanks

Suggestions for Writing Completion (or

Supply) Items

If possible, items should require a single-word answer/brief and definite statement.

The answer required is factually correct.

Omit clue/key words Don’t eliminate so many elements that the sense of the content is impaired.

Write the blank near the end of the sentence rather than near the beginning.

This will prevent awkward sentences.

If the problem requires a numerical answer, indicate the units in which it is to be expressed (pounds, inches, etc.)

Multiple-Choice Items

Multiple-choice items  Measure behavior at higher levels of taxonomy (ex:

comprehension +)  most difficult of the objective items to construct  Should be used with caution on younger children

Faults included:  Grammatical cues or specific determiners  Multiple defensible answers  Unordered option lists  Stem clues  Opinionated statements  Lack of a problem statement in the stem  Redundant wording  Wordiness in the correct option  Use of “all of the above”  Indiscriminate use of “none of the above”

Section III - Multiple Choice

They are unique in that they enable the teacher to measure at the higher levels of taxonomy as well as at the basic

knowledge level.

Instructions: Consider the following and use common sense to determine which are good (G) and which are poor (P).

 1-grammatical clue “an” eliminates options a, b, and d - better written

Grant was a/an and there are multiple defensible answers! Better

written … Who was elected president after the Civil War? A) U.S.

Grant B) Andrew Johnson C) Abraham Lincoln D) Andrew Jackson

 2-G – always arrange dates in chronological order

 3-stem clue – eliminate the word free

 4-opinion- rewritten better is “The USDA states that the principal ….”

 5-more than one defensible answer

 6-gramatical clue – are – make substitute is/are AND always use this

format sparingly AND, italicize NOT

 7-eliminate redundancy in options, also length of options is a giveaway

and test-wise students will be able to guess correctly – avoid where

correct answers are 1.5 times the length of incorrect options

 8-“none of the above” should be used sparingly, test-wise students

tend to choose “none of the above” and the correct option is e

 9-G

Multiple Choice - Key

Suggestions for Writing Multiple-Choice Items for

Higher-Order thinking 1-use pictures and graphs to measure higher-order knowledge

2-use analogies that demonstrate relationships among terms

(Physician is to humans as veterinarian is to … a) fruits b) animals c) minerals d)

vegetables)

3-require usage of previously learned principles/recall analogies (figuring miles per

gallon, perimeter, etc.)

4-the stem should formulate “a problem”; the response needs to be short but include

only the material needed to make the problem clear – don’t add extraneous information

5-be sure there is only one best choice

6-wrong answer choices should be plausible

7-eliminate grammatical clues, keep length equal, rotate the position of the correct

answer

8-include four or five options to minimize guessing

9-avoid “all of the above” and use “none of the above” sparingly

Matching Items

Matching Items  fairly easy to construct  uses reversal of options and descriptions  BUT…

 May lack of clarity/specificity in directions  Dissimilar and non-ordered lists

Faults include  Non-homogeneous lists (ex: list with states, capitals,

cities)  Improper ordering of lists (ex: mixing up descriptors with

indicators)  Easy guessing – should be more answers than questions  Poor directions – should ID bases of matching  Too many responses  Ambiguous lists – should use first & last names

Matching – Analyze the following:

__1. Lincoln a) President during the twentieth century

__2. Nixon b) Invented the telephone

__3. Whitney c) Delivered the Emancipation Proclamation

__4. Ford d) Recently resigned from office

__5. Bell e) Civil rights leader

__6. King f) Invented the cotton gin

__7. Washington g) Our first president

__8. Roosevelt h) Only president elected for more than two terms

__1. A president not elected to office a) Gerald Ford

__2. Delivered the Emancipation Proclamation b) Thomas Jefferson

__3. Only president to resign from office c) Abraham Lincoln

__4. Only president elected for more than two terms d) Richard Nixon

__5. Our first president e) Franklin Roosevelt

f) Theodore Roosevelt

g) George Washington

h) Woodrow Wilson

__1. Invented the cotton gin a) Alexander Graham Bell

__2. One of his inventions was the telephone b) Henry Bessemer

__3. One of his inventions was the telegraph c) Thomas Edison

d) Guglielmo Marconi

e) Eli Whitney

f) Orville Wright

Martin Luther King would be presented in T/F or M/C format since there are no other

civil rights workers listed.

A better constructed layout

Faults of matching:

1-lists that are not homogeneous – the above contains presidents,

inventors, and a civil rights leaders. Three separate matching exercises

should be developed. This better helps student eliminate implausible

options, and three separate matching exercises would enable the teacher

to provide better distractors, eliminate guessing, etc.

2-wrong order – columns should be reversed (description first/options

second). The student should read the longer description then glance

down the list of names. As it is written the student reads the name

Lincoln and then has to read through a lengthy list for the right answer

3-easy guessing – same number names as descriptions is a no

no….there should be at least three more names – this cuts down on

guessing

4-too many correct responses for “president during the twentieth century”

(Ford, Nixon, Roosevelt) … or does Ford mean Henry Ford, the inventor?

5-In the directions, specify if each option is used once or more than once

Constructing essay items

 Where possible, use restricted range rather than extended range – and use a “box” for the answer (1/3 of a page, 1/2 of a page, etc.). Students must restrict their answer to the text box provided-this will eliminate “bluffing”

 Use a pre-determined scoring rubric

 Implement the scoring rubric consistently with all students

 Remove or cover names to control for bias

 Score all responses to one item before scoring the next item

 Keep scores from previous items hidden when scoring subsequent items

 Use essay items when test security is in question

 Use essay items when few test items are necessary

 Use essay items when high-level cognitive processes cannot be fully measured with objective items … using words as predict, give reasons for, compare and contrast, NOT who, when, what

Gender and Racial Bias in Test Items

To avoid gender and/or racial biases in test items –

 avoid using stereotypes

 be sure to make equal reference to both

males and females and to various ethnic

groups

 balance role and power references

Recap of the class

26

1. Begin writing items far enough in advance that you will have time to revise them.

2. Match items to intended outcomes at the proper difficulty level to provide a valid measure of instructional objectives. Limit the question to the skill being assessed.

3. Be sure each item deals with an important aspect of the content area and not with trivia.

4. Be sure that the problem posed is clear and unambiguous.

5. Be sure that each item is independent of all other items. The answer to one item should not be required as a condition for answering the next item. A hint to one answer should not be imbedded in another item.

6. Be sure the item has one correct or best answer on which experts would agree.

7. Prevent unintended clues to the answer in the statements or question. Grammatical inconsistencies such as a or an give clues to the correct answer and will help those students who are not well prepared for the test.

8. Avoid replication of the textbook in writing test items: don’t quote directly from textual materials.

What’s next?

 Complete the assigned reading

 Answer the discussion question

 Complete the writing assignments

27

References

Kubiszyn, T., & Borich, G. D. (2016). Educational testing &

measurement. Hoboken, MJ: Wiley & Sons.

Payne, D. A. (2003). Applied educational assessment. Belmont, CA:

Wadsworth/Thomas Learning.

Popham, J. W. (2014). Classroom assessment: What teachers need

to know. Upper Saddle River, NJ: Pearson Education Inc.

28