Project Plan 12 pages

This handout explains 1) what assessing improvement is and isn’t, 2) why the ability to assess improvement is so important for teachers, and 3) how to plan assessments that clearly assess improvement (and what kinds of assessments don’t assess improvement).

* HOWEVER, before all that, the most important thing for you to know is that you probably don’t know how to assess improvement right now (but will believe that you do). From my experience, many of you (students) who don’t know how to assess improvement will be confident that you do, and this overconfidence will get in the way of learning this skill, and will cause you (and me) repeated frustration. I have taught over 1000 students to assess improvement (sometimes unsuccessfully), and at first, the vast majority of students write assessments that are nowhere close to assessing improvement—but the students are sure that they do assess improvement. We have two types of thinking in our minds—System 1 thinking and System 2 thinking. System 1 thinking is more intuitive and judges things at a glance, and helps us out when we need fast answers, but also is the source of lots of bias, illogic, and errors. System 2 thinking is that more slow and effortful and systematic logical thinking, and it’s this System 2 thinking that most of you will need to use in order to write assessment plans that truly assess improvement.

What Assessing Student Improvement IS and ISN’T

Assessing improvement means assessing whether or not students have gotten better or more consistent than they were before in their knowledge, skills, habits, etc. Do students know more (knowledge) at a later assessment than they did earlier in the week, month, or year? Can students do something better (a skill or performance) at a later assessment than they did earlier in the week, month, or year? Do students do some positive habit more consistently now than they did earlier in the week, month, or year? Assessing improvement is about comparing where students are at now to where they were before, and seeing if they have gotten better, worse, or stayed the same.

Assessing improvement does NOT mean assessing whether students are good or bad at something. Let me say that again—Assessing improvement does NOT mean assessing whether students are good or bad at something. A student who is good at something (e.g., writing stories) may have not improved at all in that outcome this year, but a student who is mediocre at it may have improved a lot on that outcome in the last month. In that case, the second student met the goal of improvement but the first one didn’t. Assessing improvement also does NOT mean assessing whether or not students have met some fixed and arbitrary level of performance (e.g., can identify 80% of sight words). When assessing improvement, we don’t focus on something such as whether students could read 80% of sight words, but we would focus on whether they can read more words now than they could in the past. Finally, assessing improvement does NOT mean assessing whether students are better than other kids or “on grade level.”

Why is it Important for Teachers to be Able to Assess Student Improvement?

First, one of the central jobs of teachers is to help students improve their skills and habits, but if teachers are not skillful at assessing student improvement, it is difficult for teachers to assess their success in reaching this goal, and to know when they need to try something different.

You might wonder, doesn’t all assessment assess what students have learned, and areas in which they have improved? In fact, a great deal of classroom assessment simply assesses what students currently know and can do, but doesn’t tell us whether they are more knowledgeable or more skillful or have better habits than earlier in the day, month, or year. Much of classroom assessment simply determines whether children can or can’t do whatever was the objective in the lesson plan, and this kind of “sorting” assessment creates problems for students at every ability level.

To know whether they are reaching all students, teachers need to know (in part) whether those students are improving in their skills and habits, regardless of their current level of performance, and regardless of whether those students’ performance is below average, average, or above average. When assessing improvement, you don’t care whether the student is below or above average at the ending assessment or whether they have met some arbitrary/fixed objective or standard—you care about whether they have made progress or improved over their earlier performance. (Yes, I’m repeating myself a bit: Trust me, it’s intentional.)

Second, and related to the first point, frequent use of assessment that merely classifies students, or that results only in grades or rankings relative to one’s classmates, without identifying and recognizing improvement, can create profound problems for student motivation and achievement. In simple terms, very able students can often continue to “do well” in a class without trying very hard or improving as much as they might. Less able students can often continue to be viewed, and view themselves as below average in skills, although they are trying their hardest, and making notable improvements. In classrooms where success is viewed in terms of relative ability or grades (rather than improvement), many able students learn that a great deal of effort is not needed for success while less able students may learn that expending effort does not lead to success (since success is defined not as steady progress but as being better than average). Over time, both types of students often stop working and improving as much as they could, because the most advanced students don’t need to work much to meet the standard and many of the least advanced students have learned that no matter how hard they work, they will be labeled as failures. Unwittingly, the way we assess students (based largely on relative ability, not progress) leads many more able students to view hard work as unnecessary for success, and leads many less able students to see hard work as not leading to success—thus undermining the work ethic in both groups.

Assessing students in terms of whether they met universal and fixed standards/objectives (instead of assessing improvement) also contributes to students developing a “fixed” view of ability, the belief that ability is something you just have or don’t have—rather than believing that ability and intelligence can be improved through effort (which is the healthier view). People who adopt the fixed view of ability set less challenging goals for themselves, tend to give up when the going gets tough, tend to focus on performance goals (proving ability) rather than learning goals (improving ability), and tend to blame others when they fail, rather than focusing on what they can do to improve. Also, many of the goals we value most are complex skills and performances and crucial habits that improve gradually over long stretches of time, and if we aren’t often assessing for improvement, we are probably neglecting some of our most valued educational goals. In sum, the way we frequently assess young children (based on fixed objectives or sorting kids into high and low groups/performances) creates a range of predictable problems for students’ learning and development.

However, where teachers are focused on effort and progress, lower-performing students are recognized for effort and progress, while even the highest performing students are challenged to do better, if they are not making adequate effort or improvement. In order to emphasize effort and progress, teachers simply must know how to assess student improvement.

BAD or Incomplete Examples

BAD Plan #1 for Assessing Improvement

(This does NOT assess improvement—it is a BAD example)

Objective: Students will improve their ability to describe their feelings during actual classroom conflicts.

a. I will observe the students from September through November, and for each student conflict observed, will document on a checklist which student(s) described their feelings, and which students did not.

b. A student will have met the objective if s/he described their feelings in at least 75% of the conflicts recorded in November.

c. A student did not meet the objective if s/he described their feelings in fewer than 75% of the conflicts recorded in November.

Note: This never compares later performance to earlier performance, so it simply can’t assess improvement. Also, if kids were at 85% to begin with, hitting the 75% mark is going backwards, not improvement. Similarly, kids could have gone from 0% earlier in the year to 65% now, and be rated as not improving. Fixed targets such as 4 out of 5, or 75% accuracy can NEVER be relied on to assess improvement.

BAD Plan #2 for Assessing Improvement

(This does NOT assess improvement—it is a BAD example)

Objective: Students will improve their ability to describe their feelings during classroom conflicts.

a. I will observe the students from September through November, and for each student conflict observed, will document on a checklist which student(s) described their feelings.

b. A student will have met the objective if s/he improved their ability to describe their feelings.

c. They didn’t meet the objective if s/he didn’t improve their ability to describe their feelings.

Note: This is circular reasoning—you’re sort of saying you’ll know they got better if they got better, but never say what the indicators of improvement are. Also, in steps b & c, there is no comparison of the later performance to the earlier performance, so this can’t assess improvement over time.

BAD Plan #3: An INCOMPLETE Plan for Assessing Improvement

(This gives one example of how a student might improve but does not address what improvement would look like in general—it is an INCOMPLETE assessment plan)

Objective: Students will improve their ability to describe their feelings during actual classroom conflicts.

a. I will observe the students from September through November, and for each student conflict observed, will document on a checklist which student(s) described their feelings, and which students did not.

b. A student will have met the objective if s/he described their feelings in 25% of the assessments in September and at least 75% of the conflicts recorded in November.

c. A student did not meet the objective if s/he described their feelings in fewer than 75% of the conflicts recorded in November.

Note: We want you to be tuned into any of the ways in which improvement might occur, especially for students whose ending performances are at very different levels, so it is inadequate to just describe one way in which improvement might occur. For example, a student might be at 0% in September and 10% in November and that also would be improvement. Part of what I’m trying to teach you here is to stop focusing on some fixed ending performance target, and focus instead on childrens’ growth trajectories over time.

BAD Plan #4: An INCOMPLETE Plan for Assessing Improvement

(This has the right pre-post structure, but is too vague in terms of what is being assessed to provide good evidence of assessing improvement.)

Objective: Students will improve their ability write creative fictional stories.

a. I will collect two fictional stories that students write in September and two fictional stories that students write in November. I will rate all four stories on a checklist I created to assess creative writing.

b. A student will have met the objective if their two November stories averaged more checkmarks than their two September stories averaged.

c. A student will have not met the objective if their two November stories averaged fewer or the same number of checkmarks compared to the average of their two September stories.

Note: At first glance, this is perfectly structured to assess improvement because it has pre-post comparisons, but it is far too vague about what is being assessed as evidence of creativity. What’s on the checklist, and what does it prove to have more checkmarks? You need to figure out what you mean by creative writing and identify clear indicators of creativity that they would need to do more of or do better in the second assessment than in the first—in order to prove improvement.

How Can Teachers Assess Student Improvement?

Enough of the negative examples! To assess improvement, you need three things:

First, you need an indicator or indicators of the student outcome you are assessing (for example, more interesting characters or more suspenseful plot might be indicators of improved fiction writing). Again, saying you’ll know their writing is better if their writing got better is called circular reasoning—it proves nothing because it doesn’t say WHAT needs to be better the second time than the first time—in order to prove that their writing improved. For many student outcomes, it takes some thought to come up with reasonable indicators of the outcome/objective.

Second, you need a pre-assessment and post-assessment because this will allow you to compare their current knowledge, skills, habits or complex performances to their earlier knowledge, skills, habits or complex performances. That is, you’ll need to compare their current performance to their performance on a similar task some days, weeks, or months ago. It is IMPOSSIBLE to assess improvement without at least two assessments that are separated in time, so any assessment plans that don’t mention a pre-assessment and post-assessment in step a, and compare the pre-assessment and post-assessment performance in steps b and c will be returned as incomplete.

Third, you need some way to collect and record data (e.g., anecdotal notes, checklists, individual science journal that students write in, samples of their writing, drawing, etc.). Grades are poor evidence, because they don’t answer the question of what exactly students were graded on, so they are too vague to assess improvement in specific knowledge, skills, or habits.

Finally, for me to know that you really understand assessing improvement, I need clear data that you know how to a) collect relevant data, b) identify what would be evidence of improvement, and c) identify what would be evidence of non-improvement. For this reason, I ask that all assessment plans be written in these separate a, b, c steps, not as just one large paragraph.

GOOD Assessment Plan #1 (for assessing increased knowledge)

Objective: Students will increase their knowledge about the arctic animal of their choice.

a. At the beginning of the unit, after each student has chosen an animal they wish to study, I will ask them to write in their science journal all the facts they know about the arctic animal they have chosen. At the end of the unit, I will again ask them to write in their science journal all the facts they know about their arctic animal. I will compare the number of facts the students knew about their animal at the end of the unit to the number of facts they knew about their animal at the beginning of the unit.

b. Students will have increased their knowledge of the arctic animal of their choice if they wrote more facts about their animal at the end of the unit than they wrote at the beginning.

c. Students will have NOT increased their knowledge of the arctic animal of their choice if they wrote the same number or fewer facts about their animal at the end of the unit than they wrote at the beginning.

GOOD Assessment Plan #2 (for assessing improved understanding)

Objective: Students will improve their understanding of how extrinsic rewards often backfire for promoting healthy development. (This is an objective for teacher candidates, and wasn’t written for young children)

a. On two occasions (before and after reading the course materials on motivation), I will ask students to write short essays explaining how extrinsic rewards often undermine intrinsic motivation. I will compare these essays in terms of the degree to which they explain the role of extrinsic rewards in interfering with meeting learners’ basic needs, the ways in which rewards get learners to focus on performance (and the reward) rather than learning itself, the negative consequences that often result from using extrinsic rewards, and the conditions under which extrinsic rewards are most likely to cause such harm.

b. I will conclude that teacher candidates deepened their understanding of how extrinsic rewards often backfire for promoting healthy development if their post-assessments were more detailed or coherent in explaining one or more of the following:

- how extrinsic rewards interfering with meeting learners’ basic needs,

- how extrinsic rewards promote performance goals and a focus on getting the reward, rather than learning goals,

- the negative consequences that often result from using extrinsic rewards, or

- the conditions under which extrinsic rewards are most likely to cause such harm.

c. I will conclude that teacher candidates did NOT deepen their understanding of how extrinsic rewards often backfire for promoting healthy development if their post-assessments were NOT more detailed or coherent in explaining one or more of the following:

- how extrinsic rewards interfering with meeting learners’ basic needs,

- how extrinsic rewards promote performance goals and a focus on getting the reward, rather than learning goals,

- the negative consequences that often result from using extrinsic rewards, or

- the conditions under which extrinsic rewards are most likely to cause such harm.

GOOD Assessment Plan #3 (for assessing improvement on a skill)

A skill means you are able to DO something (other than answer a fact-based question or perform on a fact based task), so an improved skill means you learned how to do something better than you used to be able to do it. In the assessment below, we are focused simply on getting children to describe their feelings more often during conflicts.

Objective: Students will improve their ability to describe their feelings during conflicts.

a. I will observe the students during two recording periods; 1) from September through October, and 2) from January through February. For each student conflict observed during these two recording periods, I will document on a checklist which student(s) described their feelings, and which students did not. For each student, I will compare the percentage number of times s/he describe their feelings during actual classroom conflicts in the first recording period with the same percentage for the second recording period.

b. A student will have met the objective if s/he described their feelings in a larger percentage of conflicts during the second recording period than during the first recording period.

c. A student will NOT have met the objective if s/he described their feelings in the same percentage of conflicts during the second recording period than during the first recording period, or in a smaller percentage of conflicts during the second recording period than during the first recording period.

Notice that the above assessment plan is focused on quantitative change, or the frequency of an action. However, improvements in skills are often best described in terms of improved quality of performance, as the assessment below does.

GOOD Assessment Plan #4 (for assessing improvement on a skill)

Objective: Students will improve their ability to describe their feelings during conflicts.

a. I will record how students describe their feelings during two conflicts, one in the fall, and one in the spring. For each student, I will record the exact words they used to describe their feelings during the conflicts.

b. A student will have met the objective if s/he described their feelings more clearly and in greater detail during the second conflict (in spring) than during the first conflict.

c. A student will have not met the objective if s/he did not describe their feelings any more clearly or in any greater detail during the second conflict than during the first conflict.

GOOD Assessment Plan #5 (for assessing improvement on a complex performance)

Objective: The parent will improve their parenting.

Note: As with judging understanding, complex performances are complex and there are multiple indicators of excellence that could improve, get worse, or stay the same. I’ll just pick some of the indicators of good parenting, so you get the idea of assessing complex performances. Basically, if some of the indicators get better over time and the others stay the same, they improved, or if more indicators improve over time than get worse over time, they improved on the complex performance.

a. I will shadow the parent during one week in March and one week in May. During both months, I will keep track of the amount of sleep the parent’s children get, how often they were served healthy foods, how often the parent read to the children or did other enriching activities with them (e.g., play, conversations), and how responsive the parent was to the children’s cues and interests. I will compare the results from the March and May data collection.

b. I will conclude that the parent improved her parenting if s/he was better in May than s/he was in March on more of the following indicators than s/he was worse on:

- frequency of children getting the recommended hours of sleep, and/or

- frequency of healthy meals, and/or

- frequency of reading to children or other enriching interactions, and/or

- responsiveness to child’s cues and interests, or any other possible indicators.

c. I will conclude that the parent did not improved his/her parenting if s/he didn’t improve on the following indicators from March to May or if she was worse in May than she was in March on more of the following indicators than she was better on:

- frequency of children getting recommended hours of sleep, and/or

- frequency of healthy meals, and/or

- frequency of reading to children or other enriching interactions, and/or

- responsiveness to child’s cues and interests, or any other possible indicators.

Of course, there are other indicators of good parenting, but I think you get the idea—for complex performances, if learners improve over time on more of the key indicators of excellence than they got worse on, then they improved at that complex performance. If you only listed one indicator that you are assessing (e.g., responsiveness to the child), then you would no longer be assessing the complex performance of good parenting, you would then be assessing the narrower (but important) skill of being responsive to your children.

GOOD Assessment Plan #6 (for assessing improvement on a habit)

Goal: Our son will more consistently make his bed.

Because a habit refers to a pattern of action over time, for assessing habits, you need several observations/pieces of evidence at the pre-assessment, and several observations/pieces of evidence at the second/post-assessment, later on. A skill means you CAN do it, a habit means you consistently DO IT, so habit assessments focus primarily on the consistency with which the person does something.

a. I will record on the calendar the number of times Johnny makes his bed (without reminders) during the first full week in March and during the first full week in May. I will compare the numbers of times he made his bed in that week in May to the numbers of beds he made in that week in March.

b. If Johnny made his bed more times in the first full week in May than he did in the first full week in March, then the habit of making his beds increased or improved.

c. If Johnny made his bed the same number of times or fewer times in the first full week in May than he did in the first full week in March, then the habit of making their beds did not increase or improve.

Assessing Emotions, a special note. Unless a child has had emotional problems, assessing increased enjoyment, excitement, happiness, or pride is not necessary. If a child truly enjoys learning something, that is good enough—most of the time, there is no need to assess whether they enjoyed it more than before. Of course, if there were problems with student motivation in a classroom, or motivation and attitudes were declining, assessing an increase in positive emotions would be useful, but such assessments are not needed for the assignments in this class.