mod3-prep-eval

DATA-DRIVEN TEACHERS

Dr. Scott McLeod, Director

What is Data-Driven Decision-Making? Data-driven decision-making (DDDM) is a system of teaching and management practices that gets better information about students into the hands of classroom teachers. Many teachers reject the idea of DDDM because of its association with the federal No Child Left Behind Act (NCLB). This is unfortunate, because a multitude of schools and districts across the country are seeing substantial improvements in student learning and achievement as they incorporate data-driven practices. Teachers in these schools are finding that intelligent and pervasive uses of data can improve their instructional interventions for students, re-energize their enthusiasm for teaching, and increase their feelings of professional fulfillment and job satisfaction. This white paper outlines many of the basic competencies that teachers need to be effective data-driven classroom instructors. These competencies were developed in conjunction with the International Society for Technology in Education as part of a comprehensive professional development initiative for the Chicago Public Schools Office of Technology Services eLearning and are shared with permission. Other school systems are invited to use this competency framework (also available in table form) to structure their own training initiatives for instructional staff. Essential Concepts One of the most important tasks for educators is to understand the differences between DDDM and NCLB. Data-driven decision- making is about getting better information into the hands of classroom instructors; NCLB is about accountability to the federal government for the education money it sends to the states. Educators should be careful not to reject DDDM principles and practices, which have been shown to have positive impacts on student learning and achievement gaps, because they are angry about federal and state NCLB implementation decisions. Data-driven activities existed in some schools long before NCLB was passed and will continue in many schools regardless of what happens with the federal legislation. Data-driven educators should be able to articulate the essential elements of effective data-driven education outlined in the diagram below. The five major elements of data-driven instruction are:

• good baseline data, • measurable instructional goals, • frequent formative assessment, • professional learning communities, and • focused instructional interventions.

These elements interact to enhance student learning and to inform teacher practice.

Teachers Data-driven teachers understand the importance of utilizing multiple measures, and multiple indicators within measures, when assessing school and student success (Bernhardt, 2004). For example, data from a single administration of a statewide reading test do not give teachers the information they need to improve student learning. Information from other assessments, measures of student engagement, previous programmatic interventions, and other data are needed for teachers to design appropriate instructional interventions. Similarly, use of a single formative assessment to measure students’ reading progress is not as reliable as using multiple, different assessments to triangulate on the complex concept of student reading. Data-driven teachers need to be savvy consumers of

http://www.hved.org/web-content/PDFS/Data%20Driven%20Teachers.pdf

summative assessment data, such as those from yearly state tests, who understand when and how the data can, or can't, inform teacher practice. Collecting and Analyzing Summative Data Educators in data-driven school organizations are expected to utilize data from yearly summative assessments to improve student learning. For example, teachers already help administer those tests; they also should be able to get relevant summative test data out of district data management and analysis systems (e.g., student information systems, data warehouses) for baseline analytical and reporting purposes. In other words, educators need to be able to get their hands on the data from yearly summative assessments that will help them improve instructional practice. Access to the raw data is crucial, because educators invariably want more detailed data, or want data presented in different ways, than paper reports typically provide. Teachers Once classroom teachers have access to good baseline information, they should work with their administrators to select key indicators of success for their classrooms. In order to do this, teachers need to be well-grounded in assessment literacy concepts so that they can appropriately interpret summative baseline data. Teachers also need to give ongoing feedback to building- and district-level administrators about the usefulness of the data and/or reports that they are receiving. Setting Measurable Goals Once armed with key summative indicators of classroom success, educators can use those baseline data to identify mastery levels and learning needs of classes, demographic subgroups, and individual students. Data-driven educators then use that information to set measurable year-end instructional goals, which serve as meaningful targets to guide their pedagogical strategies. These goals are often referred to as SMART goals. The acronym stands for Specific, Measurable, Attainable, Results-Oriented, and Time-Bound. An example SMART goal might look something like the following:

The percentage of third grade students scoring at Level 3 or higher on the state mathematics test will increase from 64% in Spring 2004 to 82% in Spring 2005. Focus areas for improvement

1. Number sense 2. Computation 3. Measurement

Data-driven educators recognize that formalized goal-setting can lead to improved student learning outcomes. All SMART goals created by teachers and administrators should have the following six components (with example language from the SMART goal above):

1. A measurable baseline (64%); 2. A measurable target (82%); 3. A specific time frame (Spring 2004 to Spring 2005); 4. Specificity about what is being assessed (percentage of third grade students scoring at Level 3 or higher); 5. Specificity about the method of assessment (the state mathematics test); and 6. Focus areas that guide future action needed to reach the learning target (number sense, computation, and measurement).

Inclusion of these six components ensures that SMART goals meet the criteria represented by the acronym. SMART goals can be used with common assessments, teacher-made rubrics, and other types of assessments as well as with standardized tests from publishing companies and state departments. Teachers Data-driven teachers identify and work toward only a few key instructional goal areas each year. Teachers often are overwhelmed by the multitude of learning needs present in their classrooms and must combat natural tendencies to either create too many goals or to become discouraged and shun goal-setting altogether. Teacher goal-setting should address instructional areas that are both important and strategic. Remembering the Pareto Principle that 20% of activity causes 80% of results is critical at this stage of the DDDM process. Evidence from successful data-driven schools shows that strategic focus and success in a couple of key areas commonly carries over and alleviates other instructional and behavioral concerns as well. Collecting and Analyzing Formative Data As noted above, data-driven schools have a good sense of where their students are at the beginning of the year and have measurable goals for where they want their students to be at the end of the year. The next step for educators is to implement a system of frequent formative assessments in order to benchmark the progress of their students during the school year toward those year-end goals. Simply using baseline data to set measurable year-end goals, without also implementing a system that allows for frequent analysis and adjustment of instructional and organizational practice, is not likely to result in significant improvements in student learning.

http://www.hved.org/web-content/PDFS/Data%20Driven%20Teachers.pdf

Effective formative assessment practices, implemented during the school year, have been shown to be a powerful mechanism for improving student learning. Research meta-analyses have shown that good formative assessment has a greater impact on student learning, and on achievement gaps, than any other instructional practice (see, e.g., Black & Wiliam, 1998). To realize the instructional power of formative assessment practices, educators also need the opportunity to meet regularly and frequently to have collaborative, data-based discussions about student progress. During these meetings, educators identify emergent patterns from the formative data and discuss what the data tell them about students’ progress toward year-end learning goals. Teachers can then collaboratively identify appropriate instructional interventions that can be implemented during the next instructional cycle and collectively commit to implementing those interventions. These types of professional learning communities have been shown to have major impacts on student achievement and teacher satisfaction. Teachers Data-driven teachers utilize their instructional expertise to identify key formative indicators of success that can be used to measure student progress during the school year. They also use appropriate technologies to collect, organize, analyze, and report that data to students, parents, administrators, and other teachers. Other key skills of data-driven teachers include knowledge of relevant assessment literacy concepts (in order to appropriately interpret formative assessment data), the ability to engage in root cause analysis to identify appropriate instructional interventions, and the capacity and willingness to work effectively with other staff on shared instructional problems and solutions. Making Changes Data analysis is meaningless if it does not result in meaningful instructional change. Data-driven educators are able to use summative and formative assessment data together to implement strategic, targeted, focused instructional interventions to improve student learning. These interventions should be aligned with state standards and district curricula as well as content-specific, developmentally- appropriate best practices. Teachers should work with curriculum specialists in their districts and states to identify effective, grade- level instructional practices for their subject areas. Many K-12 teachers feel disempowered and fatalistic about their ability to significantly impact student learning outcomes. Many teachers feel that the academic success of their students and schools is dependent upon the input characteristics of their students and families. Other educators, however, believe that collaboratively they can have powerful impacts on student learning. These latter schools, which are recognizing that they can make a difference and are strategically and intelligently redesigning instructional and organizational practices to support student learning, are the ones that are closing achievement gaps and succeeding in this new era of accountability. Schools that continue to struggle are those that place the bulk of the responsibility for student learning, and the blame for the lack thereof, on students and families rather than accept the fact that many classroom practices and school structures could be changed to better facilitate student achievement. Teachers Confucius noted that a journey of a thousand miles starts with a single step. While teachers may not be able to address the often- overwhelming problem of low student achievement all at once, they can take small steps that together add up to big improvements over time. One of the key legacies of NCLB will be the emphasis on implementation of pedagogical practices that have been proven successful through high-quality research. Teachers can work with administrators and content-area instructional experts to implement effective teaching practices and to design and implement teacher-driven action research projects that investigate the effectiveness of specific pedagogical strategies. Data Transparency and Safety Information transparency is a necessary condition for successful data-driven education. Data-driven decision-making practices are only possible in school climates where data are valued and visible. In many data-driven schools, graphs, tables, and other indicators of data usage permeate the school environment. Discussions about data are frequent and analysis of student data is considered to be integral to the teaching and learning process. Students and parents can be important allies in this process. Rather than serving as gatekeepers, and hindering access to student learning data, educators should strive to ensure that relevant data are accessible to parents and students in order to enlist their buy-in and support. Many times data will need to be anonymized in order to comply with data confidentiality requirements. In some schools, teachers are finding that having students track their own learning progress increases their buy-in and motivation and illustrates that significant learning growth can be achieved regardless of students’ initial starting points. Educators can use print publications such as newsletters, notes home, flyers, and other mechanisms to disseminate status and progress information on key summative and formative assessment indicators to parents and students. Electronic communication channels such as web sites, listservs, and e-mail newsletters also are useful tools to communicate with local communities. Teachers and administrators should utilize data to celebrate instructional progress and successes as well as to address continuing gaps or needs. Teachers Data-driven teachers view data as feedback, not as indictments. They use data to inform pedagogical modifications and actively seek out more data to judge the success of those changes. Data-driven teachers also are willing to discuss their instructional strengths and http://www.hved.org/web-content/PDFS/Data%20Driven%20Teachers.pdf

weaknesses with peers in order to facilitate shared communities of practice that are focused on individual and organizational learning. By recognizing and acting upon the fact that all educators, like students, have areas in which they could improve, teachers can be models of life-long learning for the students that they serve. Alignment for Results One of the most difficult challenges for teachers and administrators is making the mental paradigm shift from existing practice, which commonly emphasizes process and delivery, to a mindset dedicated to the achievement of results. Successful data-driven educators recognize that accepting greater responsibility for student learning can result in improved student learning outcomes. Results-driven educators assess all educational practices in light of their impacts on student learning. Any instructional practice, organizational structure, or school program that hinders student success is reexamined and redesigned. Even successful practices are examined to see if they can be improved. Results-driven educators understand the importance and impact on student learning of continuous and progressive improvement, and recognize that even small improvements add up over time to become large ones. This latter point is particularly important, because ambitious long-term goals like “achieving 100% proficiency” can be disabling rather than motivating. Turning desired outcomes into minute, concrete, short-term goals and then successfully achieving those goals is inherently motivating and can turn organizational inertia into desired progress. Instead of teachers individually selecting the content and direction of their professional development plans, teachers and administrators should work together to ensure that professional development opportunities are aligned to student, school, and district learning needs. Similarly, curricular design and delivery also should be aligned to meet these needs. In results-driven school systems, all processes and programs are designed to facilitate maximal student learning: the guiding paradigm is “If it’s not working, why are we doing it?” Teachers Teachers who have incorporated a results orientation into their instructional practice continually seek out evidence about the success or failure of their pedagogy. Ineffective strategies are discarded, and successful strategies are tweaked or modified to achieve even larger learning gains. Data-driven teachers exhibit a constant dissatisfaction with the status quo and continually strive for further improvement, even when already exhibiting high levels of success. These teachers also are willing risk-takers who understand that trying something new and different may be the only path to improved outcomes. Conclusion Mike Schmoker (1999) has said that if educators constantly analyze what they do and adjust to get better, student learning will improve. By focusing initially on small, rapid improvements and then building upon those toward an ongoing process of continuous reflection about classroom instruction and student learning outcomes, teachers across the country are significantly impacting student achievement. When these teachers also are able to participate in professional learning communities and collaboratively identify and implement effective, strategic instructional interventions, their schools are not only surviving this new wave of accountability but indeed thriving in it (see, e.g., Supovitz & Klein, 2003).

http://www.hved.org/web-content/PDFS/Data%20Driven%20Teachers.pdf