Instructional strategies inventory1

W hen delivering her opening-day speech to faculty at McKay K–8 School in Boston, second-year princi- pal Almi Abeyta hoped that displaying recent state

test results would “light a fire” among teachers and spark a powerful conversation about instructional improvement. Instead, teachers reacted with stunned silence, quickly fol- lowed by expressions of anger and frustration. It was the first they had heard about the prior year ’s decline in language arts scores. Almi felt as if she “had dropped a bomb” on the room. Far from igniting collaborative energy, her presentation of achievement data seemed to have squelched it.

As schools respond to external pres- sure to raise student achievement, the perils of examining data loom large. How, school leaders may wonder, do you con- vince colleagues that engaging in ongo- ing, collaborative data discussions is worthwhile? How do you discuss data and instruction without finger-pointing or leaping to conclusions? And how do you use insights gleaned from the data to make meaningful—and lasting—instruc- tional improvements?

A few years ago, we collaborated with a team of profes- sors, school administrators, and graduate students to write Data Wise: A Step-by-Step Guide to Using Assessment Results to Improve Teaching and Learning (Harvard Educa- tion Press, 2005). The book offers an eight-step approach to collaborative, evidence-based instructional improvement (see “The ‘Data Wise’ Improvement Process,” HEL, January/Feb- ruary 2006). Since then, schools all over the country have adopted the Data Wise approach. As we worked with many of them, we realized that teachers and administrators who are spearheading the Data Wise improvement process in their schools—as well as those pursuing other approaches—often

encounter similar questions and obstacles. So we set out to develop case studies of eight of these schools, documenting the leadership challenges that school leaders typically face during each step of the improvement process, as well as the strategies they use to address them.

Investing in Preparation In the first phase of the Data Wise process, Prepare (see “The Data Wise Improvement Process,” p. 2), school leaders typi- cally face two critical challenges: communicating the need for a data initiative and creating data teams that are equipped to

lead the work. The leaders we studied con- front these challenges in two ways: by mak- ing data relevant, and by giving their data teams time to develop the skills and sys- tems they need to be successful.

Make data relevant. As school lead- ers embark on the improvement process, they need to convince staff that looking at data will not be yet another distraction from their work but will help them do that work more efficiently. For instance, when

taking the helm of Newton North High School in Newton, Mass., a school with a history of high academic achievement, first-time principal Jennifer Price found herself in a situa- tion where test scores could easily be dismissed as beside the point. She decided to focus on a topic of longstanding con- cern to both faculty and the community: how to close the school’s academic achievement gaps. This helped her recruit a large, diverse team of faculty members to gather and analyze data. Explaining her decision to make data relevant, Jen says, “Every department sees the achievement gap manifested in one way or another. By focusing the work of the data team on the achievement gap, the use of data becomes connected to why people come to work.”

January/February 2008 Volume 24, Number 1

Leadership Lessons From Schools Becoming “Data Wise” by Jennifer L. Steele and Kathryn Parker Boudett

a closer look

Mentors Help Prepare New Principals 4 conversation

Robert Pianta on Measuring Teacher Quality 8

I n s I d e

Make data relevant

and give data teams time

to develop the skills they

need to be successful.

1

Published by the Harvard Graduate School of Education

HArvArD EDucATIoN LETTEr

Set aside time to build capacity. In addition to estab- lishing data teams, school leaders need to give team members time to develop their knowledge and to cre- ate systems that support the team’s efforts (see “Is Your School ready to Become ‘Data Wise?’” p. 3). Shortly before undertaking the Data Wise improvement process, Pond cove Elementary School in cape Elizabeth, Maine, had emerged from a cumbersome, externally imposed assessment initiative that was ultimately suspended. Prin- cipal Tom Eismeier knew that if the Data Wise approach was to be successful, he and his data team would have to think carefully about how to get the process right. As media specialist Shari robinson recalls, “[We] didn’t want it to end up as just another failed initiative.” conse- quently, Tom, Shari, and the rest of the data team spent a semester in preparation. They took inventory of data already in use at the school, developed a computer-based data analysis system that would be easy for teachers to use, and chose an instructional focus—literacy—that the faculty had already made a priority for the year. Although the team often felt they were losing a race against the clock as time wore on and the most recent test data grew stale, their patience paid off in the end, when their user- friendly approach to data analysis was well received by their colleagues.

Facilitating Large-Scale Inquiry In moving from the Prepare to the Inquire phase, school leaders often face another critical challenge: how to engage the entire faculty in honest conversations about data, particularly when, as Shari robinson puts it, “Data can wound.” This was the challenge Almi Abeyta encountered in presenting her data to the McKay School faculty. In addressing that challenge, Almi and other leaders we observed demonstrate two important lessons: establish clear norms for looking at data, and conduct frequent, focused conversations about student learning.

Establish clear norms for data analy- sis. At McKay, Almi bounced back from her initial presentation and learned to lead productive data conversations by creating a transparent, nonthreatening discussion process. Adapting a protocol commonly used to analyze visual art, she and her data team now present test score data graphically during faculty meetings and ask teachers to ground their data interpretations in objective observations. With its focus on observa- tion and objectivity, this approach facil- itates rich conversations and minimizes the threat of finger-pointing or blame.

Conduct frequent, focused conversa- tions about student learning. At Mur-

phy K–8 School in Boston, principal Mary russo and her staff also rely on clear norms to promote inquiry. They have developed a structured peer-observation protocol in which the teacher who is being observed chooses the les- son, briefs colleagues beforehand, and specifies the aspects of the lesson on which she would like feedback. This pro- tocol puts teachers at ease during the potentially threaten- ing experience of being observed by their colleagues and makes it easier to conduct peer observations on a regu- lar basis. Murphy second-grade teacher Tricia Lampron recalls the first time she participated in this process: “If there were no steps or predesigned process, I wouldn’t have known how to prepare or what my peers would be watching. But the structured process provided an oppor- tunity to focus the observation. . . . That made all the difference.”

Taking Meaningful Action In moving into the Act phase, Data Wise leaders face the challenge of helping faculty choose, implement, and assess a viable action plan based on insights from the data they have gathered. Taking action can prove difficult; fac- ulty members often have divergent ideas about how broad or narrow the action plan should be and what kinds of instructional improvements are likely to have the most impact. The schools we observed address this challenge by getting down to the “nitty-gritty” in their action plan- ning and by helping teachers “keep the faith” when refine- ments are needed.

Harvard education Letter

PubLISHeR Douglas Clayton

edIToR Caroline Chauncey

ASSISTANT edIToR Nancy Walser

PRoducTIoN MANAgeR Dody Riggs

FAcuLTy edIToR Richard F. Elmore

edIToRIAL AdvISoRy boARd Katherine C. Boles, Lecturer, HGSE; Laura A. Cooper, Asst. Super inten­ dent, Evanston Township (Ill.) High School; Linda Darling­Hammond, Professor, Stanford University; Sally Dias, Vice President for Programs and Partnerships in Education, Emmanuel College, Boston; Susan Moore Johnson, Professor, HGSE; Robert Kegan, Professor, HGSE; Peggy Kemp, Head of School, Fen­ way High School, Boston; Marya Levenson, Director, Teacher Educa­ tion, Brandeis University; Deborah Meier, Co­Prin cipal, Mission Hill School, Boston; John Merrow, President, The Merrow Report; Jerome T. Murphy, Professor, HGSE; Mary Grassa O’Neill, Lecturer, HGSE; Arthur J. Rosenthal, Publish­ ing Consultant; Catherine Snow, Professor, HGSE; Jay Sugarman, Teacher, Runkle School, Brookline, Mass.; Ariadne Valsamis, Director of Program and Resource Develop­ ment, John F. Kennedy Library Foundation.

Harvard Education Letter (ISSN 8755­3716) is published bimonthly by the Harvard Graduate School of Education, 8 Story St., Cambridge, MA 02138­3752. Sec­ ond­class postage paid at Boston, MA, and additional mailing offices. Postmaster: Send address change(s) to Harvard Education Letter, 8 Story St.,Cambridge, MA 02138­3752.

Signed articles in the Harvard Education Letter represent the views of the authors.

©2008 by the President and Fel­ lows of Harvard College. Published as a non­profit service. All rights reserved. Special permission is required to reproduce in any manner, in whole or in part, the material herein contained.

How To SubScRIbe Send $38 for individuals, $49 for institutions ($52 for Canada/ Mexico, $63 other foreign, in U.S. funds only). Subscription prices subject to change without notice. Single copies, $7.00. Back issues and bulk subscriptions available at special reduced rates.

Address all correspondence to Harvard Education Letter, 8 Story St., Cambridge, MA 02138­3752; phone 617­ 495­3432 in Massa chusetts, 800­513­0763 outside Mass.; fax 617­496­3584; email: [email protected]; web: www.edletter.org.

PR EP

A R

E IN

Q U

IR E

ACT

1Organize forCollaborative Work

2 Build Assessment Literacy

3 Create Data Overview

4 Dig into Student Data

5 Examine Instruction

6 Develop Action Plan

7 Plan to AssessProgress

8 Act and Assess

The data wise Improvement Process

2 Harvard education Letter January/February 2008

Get down to the “nitty-gritty.” When test scores at Mason Elementary School in Boston showed that students were struggling with writing about texts, teachers were shocked. After all, stu- dents wrote about texts all the time in their readers’ notebooks. However, when teachers examined the notebooks collab- oratively, they realized that each teacher had different standards for evaluating students’ reading-response letters. As in many schools, a key challenge the teach- ers faced was defining consistent instruc- tional expectations across grades. After much conversation and debate, they developed an action plan that described exactly how they would teach and assess reading-response letters at each grade level. Teacher and data coordinator Hilary Shea explains that this “nitty- gritty” focus was the key to the plan’s eventual success: “If you want improve- ment . . . you can’t tackle everything at once. Getting people to choose small topics is so important.”

Keep the faith. The Data Wise improvement process is not a one-time event but a model of ongoing inquiry. The school leaders we observed in our case studies understand that the work of continual improvement is never done. At community Academy, an alterna- tive high school in Boston, principal Lindsa McIntyre and her faculty devised an action plan for assigning homework consistently across the school. However, in assessing the plan’s implementation and effectiveness, they realized that their initial success in raising teachers’ expectations and students’ engagement was being eroded by the ongoing transfer of new students into the school, with some classes doubling in size. Some new students resisted doing homework, while others found the require- ment overwhelming and despaired of keeping up. Lindsa and her team realized they had to explore new alterna- tives: Establish a study hall? require new students to start on Mondays, so teachers could plan orientation activities? The challenge for Lindsa and the leadership team—as for any school leader at this phase of the cycle—is to take heart from evidence of success while continuing to target areas for improvement.

Learning from Leaders The leaders in our eight case studies creatively adapted the Data Wise improvement process to meet the unique chal- lenges facing their schools. At the same time, they drew many of the same lessons from their experiences, based on their common commitment to shared leadership, collabor- ative learning, and evidence-based decisionmaking.

As for Almi Abeyta, the lessons she learned from her initial presentation fueled her determination to foster pro- ductive, collaborative data conversations among the fac- ulty. Two years later, she was able to turn the opening-day presentation over to her enthusiastic data team, who pre- sented evidence of academic improvement in several areas and then announced that McKay had made Adequate Yearly Progress in language arts. on hearing the news, teachers cheered. Then they dove right into a spirited dis- cussion of how to build on their students’ progress in the coming year. n

Jennifer L. Steele is a doctoral student at the Harvard Gradu- ate School of Education. Kathryn Parker Boudett teaches at the Harvard Graduate School of Education and is the director of the Data Wise Project.

Data Wise in Action, edited by Kathryn Parker Boudett and Jennifer L. Steele (Cambridge, MA: Harvard Education Press, 2007), $29.95. To order, call 1­888­437­1437 or visit harvardeducationpress.org.

Is your School Ready to become “data wise”? If you are wondering whether your school is ready to use student data to improve teaching and learning, you may want to consider four key questions:

1. Is our principal committed to becoming a “data wise” leader? For the Data Wise improvement process to work successfully at the school level,

it is essential that the principal be on board. If you are the principal, this means that you must commit to building a culture based on trust, where teachers feel comfortable admitting what they don’t know and confident that they will be supported as they strive to improve their practice. If you are a teacher, coach, or administrator, the first step toward bringing your principal on board may be to discuss how the effective use of data might improve teaching and learning in your school.

2. Is there time for teachers to work collaboratively? Teachers need time to engage regularly in conversations with colleagues about

a wide range of data sources. Whether you rearrange your school schedule to ensure that teachers have ample common planning time or rethink the way you use existing common time, you’ll want to be sure that the insights arising in small group meetings can be shared among your entire staff.

3. Is there someone besides the principal who can oversee data management? To make collaborative time most effective, it is critical that someone at your

school—ideally not the principal—take responsibility for managing the data and ensuring that it is shared with teachers in a way that draws them into the conversation. For many schools, freeing up a teacher or administrator to work part­time for a year to create a system for collecting, analyzing, and discussing data is an investment that pays off for years to come.

4. Is there professional support for improving instruction? Finally, if you want all your data work to translate into real changes in classroom

practice, it is important to think ahead about how teachers will gain access to high­quality professional development, whether from within or outside the cur­ rent school staff. n

Harvard education Letter January/February 2008 3