Synthesis

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

48

A Mechanism to Increase Literacy and Math Skills to Reduce Summer Learning Loss

Adam Reynolds

West Rocks Middle School

Olcay Yavuz Southern Connecticut State University

There is a lack of consensus among school districts on how best to mitigate reading and math learning loss during the summer months. The purpose of this quasi-experimental quantitative exploration study was to determine if there was a statistically and practically significant effect of an educational program on summer learning loss in reading and mathematics in grades five and six students. The findings of this non-randomized controlled trial indicated that the students who participated in Summer Academy grew in reading almost double compared to those students that did not participate in Summer Academy. The findings also suggested that the students who participated in Summer Academy regressed in math almost double compared to those students that did not participate in Summer Academy. The implications indicate that school leadership is integral to the success of summer programs especially with regard to the impact programs have on economically disadvantaged students. Keywords: reading learning loss, math learning loss, summer academy, school leadership

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

49

The current American school calendar is approximately one hundred eighty days with three months off for the summer. This gap promotes learning loss in the summer months and accelerates the achievement gap (Cooper, 2003). Smith, (2012) and Hanover Research (2017) indicated that summer slide--the loss of learning over the summer break--is a huge contributor to the achievement gap between low-income students and their higher-income peers. In other words, summer learning loss during elementary school accounts for two-thirds of the achievement gap in reading between low-income children and their middle-income peers by ninth-grade High-income families supplement learning opportunities through programs while lower-income families struggle to maintain their education and experiences.

The research that investigated academic achievement over the summer months, observed a negative trend in achievement for various students. This trend has been termed “summer learning loss” (Cooper, 2003), “summer setback” (Allington et al., 2010), or “summer slide” (Slates, Alexander, Entwisle, & Olson, 2012). Even though there has been a focus on summer learning loss, there is still a lack of evidence based and research supported practices to improve summer learning opportunities for all students. This study uses the term “summer learning loss,” which is defined as the decline in achievement over the summer months when formal school-based instruction is withdrawn for most children; the focus is specific to summer learning loss in reading and math achievement. “Meghan Kuhfeld draws on data from the 3.4 million students who took the NWEA MAP Growth assessments to find that summer slide is common, but not inevitable. According to the data, the students who experienced the greatest loss were those who made the greatest gains during the previous school year” (Kuhfeld 2019).

The focus provides emphasis on the influence of summer learning loss on distinct groups. Such insights might guide future interventions to reduce gaps in opportunity and achievement. These issues provide a solid foundation for current and future research to explore the current influence and potential of summer activities to reduce gaps in opportunity and achievement.

Theoretical Framework

This quantitative study sought to determine an effective mechanism for teaching literacy and math during the summer months. Identifying risk and protective factors related to academic success has been a major area of study, as understanding these factors should provide better rationale of student success. Better understanding of risk and protective factors allow educators to increase academic success of at-risk students and better meet their academic needs (Christiansen, Christiansen, & Howard, 1997). The risk and resilience framework is supported by research of various methodologies (Corcoran & Nichols Casebolt, 2004). Recently, researchers have applied the risk and resilience framework to intervention (Corcoran & Nichols-Casebolt, 2004). The main protective factor of student distress and school drop-out indicated by the students was a more sensitive and supportive relationship with both parents and teachers. On the contrary, parents and teachers indicated as possible causes of school distress and drop-out the intrinsic students’ motivation or other external factors associated with the community (Pedditzi, Fadda, & Lucarelli, 2022) To this point, however, the risk and resilience framework has not been utilized to better understand summer learning loss. More specifically, research has not yet clearly determined

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

50

factors that may increase or reduce the risk of summer learning loss in reading or math. This is a gap in the literature with regard to the framework. Figure 1 Risk and resilience framework. Adapted from Murray (2003)

Particularly, middle school students who are struggling in literacy and math are affected

by learning loss during the summer months. According to Fifer and Krueger (2006), several scholars attribute this pattern of summer learning loss to the faucet theory. During the school year, the faucet of learning is on for all students, while during the summer it remains only for more advantaged children who continue to participate in some form of educational activity. It is vital to understand the learning loss of struggling students during the summer months. Teachers are aware of the learning loss of students throughout the summer months because during the first months of school, teachers spend a great deal of time reviewing work from the year before (Fifer & Krueger, 2006). Disadvantaged students usually have greater summer loss than students with higher socioeconomic status. Fifer and Krueger reported, “One study found that literacy and math skills of middle-income students improved over the summer, while those of low-income students deteriorated, so that a three-month achievement gap emerged during the summer.” More students with a low socioeconomic status have a greater summer learning loss; therefore, these students should have the opportunity to attend summer programs to decrease their summer learning loss. In addition, educators who teach students of low socioeconomic status should be given opportunities during the school year to attend summer learning loss workshops. The workshops should increase educators’ knowledge on how to help struggling middle school students during the summer and throughout the regular school year.

Rutter (1987) conceptualized risk and resilience as opposite poles “of individual differences in people’s response to stress and adversity.” Risk encompasses the negative circumstances that an individual faces which are associated with poor outcomes. Resilience represents the positive counterpart of vulnerability and risk factors (Werner & Smith, 1982,

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

51

1992). Resilience is a complex construct that involves the interaction between adversity and an individual’s internal and external protective factors – as well as developed competencies – that allow one to overcome adversity (Kaplan, 1999; Ungar, 2005).

Moreover, Rutter (2010) conceptualized resilience as relative resistance to environmental risk experiences and not just social competence or positive mental health. There is a universal finding of individual differences in people’s responses to all kinds of environmental hazard and evidence of ‘steeling’ effects in which successful coping with adversities lead to improved functioning in individuals (Rutter, 2010).

The broaden-and-build theory by Fredrickson (2001) predicts that positive emotions are useful in coping. A recent study conducted by Tugade and Fredrickson (2004) further expanded this theory into the realm of coping, suggesting that positive emotions guide present coping behavior. By examining psychological resilience from subjective, cognitive, and physiological angles in the three studies conducted, Tugade and Fredrickson (2004) provide greater insight into the reasons why resilient individuals are able to effectively cope with stressful experiences, whereas others facing similar conditions do not fare as well. Thus, through exploration and experimentation, adolescents may be able to build an arsenal of effective coping resources that help buffer against negative emotional life experiences (Fredrickson, 2001).

In the Asian context, Chan (2000) conducted a study to examine the effect of resilience in reducing psychological distress through positive cognitive appraisals and adaptive coping on 245 Chinese secondary school students in Hong Kong. These students were between the ages of 13 – 18. Resilience, life events, coping strategies, and psychological distress were assessed. It was found that although students with high resiliency, compared to those with low resilience, did not consider positive events as having a greater impact. They perceived that negative events had a significantly lesser impact. In addition, low resilient students reported using passive and avoidant coping strategies significantly more frequently than high resilient students (Chan, 2000).

This quantitative study examines the impacts of the Summer Academy on students’ math and language art achievement. Besides Summer Academy, other potential risk factors (e.g., free or reduced lunch status, English language learner status, special education eligibility) and protective factors (e.g., home literacy and math activities, enrichment programs) may be also related to summer learning loss. This is a gap in the literature with regard to the summer learning loss framework.

Educational Leadership and Management of Summer Programs

There are multiple studies that have documented summer learning loss among various populations (Alexander et al., 2007; Allington & McGill-Franzen, 2003) and school districts across the United States have implemented different types of programs to mitigate that loss with varying success (Keiler, 2011). More recent recommendations indicate a call to shift to robust programs that provide students with more than basic academic skill support through trips, experiences, hands-on projects, social-emotional support, and fun (Hanover Research, 2017; Mraz & Rasinski, 2007).

New research and recommendations regarding summer learning as a time for students to have experiences similar to middle-class students (Hanover Research, 2017; McCombs et al., 2011) are creating an impetus for district leaders to embrace new summer learning

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

52

opportunities. This grounded theory study explored the beliefs and actions of district-level administrators who develop and implement summer elementary experiential summer learning programs, as well as factors that influence the development, program expectations, and outcomes. Grounded theory designs allow researchers to generate a general explanation that explains a process, action, or interaction among people (Creswell, 2017). The framework for study was supported by educational and leadership theory research from Kotter (2012), and Schein (2010).

Making Summer Count: How Summer Programs Can Boost Children’s Learning outlined additional components of quality summer learning programs (McCombs et al., 2011). These included smaller class sizes, differentiated instruction, high-quality instruction, aligned schoolyear and summer curricula, engaging and rigorous programming, maximized participation and attendance, sufficient duration, involved parents, and evaluations of effectiveness (McCombs et al., 2011). McEachin et al. (2016) identified a minimum of 70 hours of instruction as the appropriate duration and recommend that teachers hired should have recently taught the grade their summer students left or will be advancing to in the fall.

Augustine et al. (2013) provide detailed recommendations for developing and implanting summer learning programs. These recommendations cover collaborative planning, curriculum and pedagogy, staffing and professional development, best practices in enrichment, enrollment and attendance, recommended schedules, and suggestions for funding and managing costs (Augustine et al., 2013). They recommend planning for enrichment from the very start, stating that leaders had multiple goals and expectations in providing enrichment activities, including improving attendance over the summer, closing the opportunity gap, social-emotional development, and supporting academics through related art or hands-on activities (Augustine et al., 2013). Recent research around how a well-designed, research-based Hip Hop-integrated strategy may complement summer learning strategies and help improve mental health outcomes for low-income middle school youth. The study determined incorporating this into a complete summer program is a potential way to mitigate learning loss (Travis, Gann, Crooke, & Jenkins, 2019).

Purpose and Research Questions

The quantitative study sought to determine an effective mechanism for teaching reading and literacy as well as math during the summer months. The following were the guiding questions for the study. The questions led and directed the research and development of a framework of best practices for middle school literacy and math summer programs.

1. What is the impact of the Summer Academy on students’ achievement? a. What is the impact of the Summer Academy on the participants’ reading score? b. What is the impact of the Summer Academy on the participants’ math score?

2. What is the academic growth between Summer Academy participants and non- participants?

a. How do the reading growth scores between Summer Academy participants and non-participants compare?

b. How do the math growth scores between Summer Academy participants and non- participants compare?

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

53

Methodology Using student achievement data from Diversity Middle School in 2019-2020, this paper examines the effects of reading and writing instruction as well as math instruction on student achievement comparing students that receive instruction during the summer months to those that do not. To address the research questions, quantitative research is used. This section is outlined as follows: (a) research design and approach; (b) setting and sample; (c) instrumentation and materials; (d) data collection and analysis; and (e) the role of the researcher. Research Design This study can be considered as quasi-experimental. In this non-randomized controlled trial, participants are allocated to an intervention but the allocation is not randomized. This study can also be considered as a quantitative exploration program evaluation because the before and after comparison involves collecting data before and after a group of participants receive an intervention (Bowling, Ebrahim, 2005). Comparing paired data makes this a useful design; therefore, this study is a retrospective quantitative approach in which an established database with student information will be used as the main source of data. These data will include two sets of student cohort scores collected in the spring of 2019 and then in the fall of the same year to determine if the intervention in place was effective to close the achievement gap and prevent summer learning loss. Achievement data will be collected and analyzed from the spring of the 2018-2019 school year and the fall of the 2019-2020 school year. Non-Equivalent (Pre-test and Post-test) Control-Group Design Experimental (Treatment) Group A within the fifth and sixth grade cohorts and Control-Group (Non-Treatment) Group B the other students within the fifth and sixth grade cohorts are selected without random assignment. Both groups are pre-tested and post-tested.

All students were given the spring and fall reading assessment through the Northwest Evaluation Association (NWEA). This test provides the Lexile level for each child as well as the Rasch Unit (RIT) score, which indicates the level of each student’s readiness to learn. The design of the study examines the reading and math growth of each student from the spring to the fall and scrutinizes what factors impact student achievement such as gender, race, language learner status, and socioeconomic status. This study has a quasi-experimental design whereas there is one group of students in each cohort that received the intervention and one group of students in each cohort that did not receive the intervention. The independent variable is the Summer Academy. Group A: Spring NWEA --------Summer Academy--------Fall NWEA Group B: Spring NWEA -----------------------------------------Fall NWEA Control group – one group with no intervention Experimental group – one group with intervention Setting and Sample

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

54

For the purposes of this study, reading and math achievement data will be used for analysis that was collected from two cohorts of students at the Diversity Middle School in Connecticut. The school is in DRG H in Connecticut. The school profile shows 703 total students with 61% of students receiving free or reduced lunch and the school qualifies for Title I funds.

This study has a quasi-experimental design in that there is one group of students in each cohort that received the intervention and one group of students in each cohort that did not receive the intervention. The group of students that received the intervention qualified for this as a result of their spring NWEA scores. If the student scored in the 25th percentile or lower based on the 2105 nationally normed data, the student was enrolled in the Summer Academy to receive the intervention. A percentile rank indicates how well a student performed in comparison to the students in the specific norm group, for example, in the same grade and subject. A student's percentile rank indicates that the student scored as well as, or better than, the percent of students in the norm group. The remainder of the cohort did not receive the intervention.

The participant scores will be from students who were students who exited fifth and sixth grade in the 2019-2020 school year. The fifth-grade cohort consisted of 245 students. Table 1 below has the student demographics of the students. The sixth-grade cohort consisted of 267 students. Table 1 Student Demographics Fifth and Sixth Grade Cohorts Demographics Percentage 5th Grade Cohort Percentage 6th Grade Cohort Male 52.2 Female 47.8 Black 17 White 22 Hispanic 55 Asian 5 Other Races 2 Special Education 13.7 Free - Reduced Lunch 64.6 English Language Learners 16.9

46.8 53.1 17 22 55 5 2

13.7 64.6 16.9

Treatment The mechanism in place is individual, small group, and whole group reading and writing instruction as well as daily student engagement in a STEM performance task. This program is the independent variable and is four hours, five days a week for five weeks and occurred during the summer of 2019. Students had reading and writing instruction as well as a STEM performance task activity in which they applied their learning to the real world through projects. There were also four field trips over the course of the five weeks.

The cohorts were divided into five homerooms and the program consisted of five teachers – two reading, two writing, and one teacher that teaches the daily STEM performance task. The schedule rotates with consistency in the amount of time on each component: forty-five minutes of reading instruction, forty-five minutes of writing instruction, forty-five minutes for work on a

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

55

STEM performance task, then an additional forty-five minutes for conferring, skill development, and independent reading. Students were then provided lunch and recess.

The expectations included emphasis on all types of literacy, science, technology, engineering, and math, which creates a transition experience for the next grade level, encouraging the sixth-grade cohort to serve as mentors to the fifth-grade cohort, and general collaboration. See the content and design of the program below. Within each homeroom, students with needs around English language acquisition had additional support.

There are also two periods of non-instructional time during the day that students in Summer Academy interact with students have been recommended for the Extended School Year (ESY) program as a result of their qualifications and needs within Special Education. ESY had 8 students that qualified in 2019, and the program provides the opportunity to students to maintain the progress made during the year. All students interact during designated times during both summer programs. Daily Schedule for Summer Academy at Diversity Middle School

8:30 – 8.40 Staff Reports (preparation time for teachers) 8:40 - 8:55 Students arrive, breakfast is served, paraeducators supervise 9:00 - 12:27 Homeroom/SEL and Expectations

Instruction - students rotate through 4 classes with aligned curriculum: Reading, Writing, STEM, Additional Literacy and Conferring

12:30 - 12:52 Lunch for Summer Academy and ESY students 12:30 Teachers dismissed, but remain for a 90-minute data team meeting

once/week beginning on week 2 (12:30 pm - 2:00 pm or 7:30 am - 9:00 am), paraeducators remain with one certified staff member to facilitate lunch and engagement/recreation

12:55 - 1:15 Engagement/Recreation for Summer Academy and ESY

1:15 Dismissal 1:30 Paraeducators dismissed

Instrumentation As an evaluation assessment Northwest Evaluation Association (NWEA) map test was utilized. The NWEA map test is a research based, norm-referenced test administered to students in a group setting in September and April of each school year. Students take the assessment on the computer, as it is an online, adaptive benchmark and progress monitoring assessment that efficiently measures oral reading fluency, literal comprehension, and foundational skills in math and reading. The reading and math tests are separate and include a combination of multiple choice and open-ended questions. The test is untimed, but the average student completes it in about 90 minutes. There is reliability and validity evidence that is pertinent in the context of accountability of this high stakes assessment. In these situations, the accuracy and consistency of classification decisions based on test scores becomes a form of validity evidence (Cronin, 2007). Reliability is a fundamental requirement of any assessment and is central to test design.

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

56

It can be defined as the consistency of achievement estimates obtained from the assessment (Cronin, 2007). Findings of a study to determine whether test administration method influence reliability demonstrated that there was not any significant difference in test scores between participants who took a computer-based test and those who took a paper-based test. The delivery mode did not have any impact on the reliability and validity of the tests administered (Öz & Özturan, 2018). Data Collection and Analysis The data is in an existing database within the Norwalk Public Schools District. After extracting data from the district office of both the fifth and sixth grade cohorts in the fall of 2020, the researcher looked to determine that summer reading and math gains occurred in both cohorts, and what factors impacted the scores. Because the Norwalk Public Schools District has many requests for research studies, it limits approval to collect data in the district to residents and staff. Approval was granted by the director of testing and accountability, so the process included a meeting to discuss the purpose of the study, research methods, and schedule. The request included a copy of the researcher’s chapter one and IRB approval. All information was acceptable, and the district granted the necessary permission. Student names remain anonymous throughout the reporting process.

Particularly, paired sample t-tests are conducted in order to determine if there was a significant difference in the students’ reading achievement scores and math achievement scores between the pre-test (Spring NWEA map test) and post-test (Fall NWEA map test) for those students that participated in Summer Academy. First, an analysis is conducted of the fifth and sixth grade cohorts that participated in Summer Academy using the reading scores. Then, the same analysis was completed to review the fifth and sixth grade cohorts that participated in Summer Academy using math scores.

Independent sample tests are conducted in order to compare growth on reading and math scores between the students that participated in the Summer Academy and non- participants. Reading and math achievement scores between the pre-test (Spring NWEA map test) and post-test (Fall NWEA map test) will show growth. First, an analysis is conducted of the fifth and sixth grade cohorts that did not participate in Summer Academy using the reading growth. Then, the same analysis is conducted comparing reading growth data of students that participated in Summer Academy and those that did not participate. Then, an analysis is conducted of the fifth and sixth grade cohorts that did not participate in Summer Academy using the math growth. Moreover, the same analysis is conducted comparing math growth data of students that participated in Summer Academy and those that did not participate. Role of Researcher As the building principal, the researcher had access to all students’ data and facilitated the program. The district provides access to the pre and post-test reading scores as well as ongoing access to grades and progress during the summer. The district’s curriculum and intervention program was communicated and the researcher had access as well as to the teachers. The

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

57

researcher was in the role one year during the time of the data collection and analysis. The past and current relationships with students and staff did not affect data collection. Limitations There is a chance the testing conditions will change from one year to the next, and as a result, there are some validity concerns. Teachers and settings change. There were no validity concerns based on the assessment used. The NWEA was used to measure growth or regression over the summer. This assessment is research-based, is nationally normed, and is used in all schools throughout the district. Conversely, there were limitations to this study. A limitation was the smaller sample size. Because of the transience of the population in the district, only data from students that were there both in the spring and in the fall could be analyzed. This number of students was not consistent. Finally, there are no ethical issues within this study. The students’ names were removed from all data sets and replaced with identification numbers. All math and reading achievement data and personal student information will remain confidential.

Findings

This finding section presents the results of the data analysis and is organized based on the research questions. The two research questions each address reading growth and math growth separately. Particularly research questions analyze growth of summer academy participants using a paired sample t-test. Research Question 1A: What is the impact of Summer Academy on the participants’ reading score?

In order to examine whether Summer Academy participants demonstrated growth in reading, the researcher conducted a paired sample t-test. Before conducting the paired sample t-test, the normality assumptions were checked for pre and post-test scores. Two cases that were extreme outliers were removed based on inconsistent effort and other factors, so 57 Summer Academy participants were used in this study. The results showed that skewness was .15 and kurtosis was .67 based on the reading growth of the participants. Considering skewness and kurtosis together, the results meet the normality requirement and are in the acceptable range.

The test resulted in a significant difference between the participants pre-test reading score (M = 184.02, SD = 16.71) and post-test reading score (M = 192.81, SD = 12.94). The participants significantly demonstrated growth in reading after participating in Summer Academy, t(57) = 6.37, p < .05. The Cohen’s d has a value of .84, which is a large effect size. Post-test results are significantly higher than the pre-test results indicating the impact of the Summer Academy on those students that participated. Table 1 T-test Statistics Comparing Student Growth in Reading for Summer Academy Participants n Mean SD SE Pretest Reading Score 56 184.02 16.71 2.22

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

58

Post-Test Reading Score 56 192.81 12.94 1.70 Table 2 Paired Sample T-test Results Comparing Student Growth in Reading for Summer Academy Participants Mean SD SE t df Sig. Pretest Reading Score - Post-Test Reading Score 8.79 10.52 1.38 6.37 57 .00 Cohen’s d=0.84 Research Question 1B: What is the impact of Summer Academy on the participants’ math score?

In order to examine whether Summer Academy participants demonstrated growth in math, the researcher conducted a paired sample t-test. Before conducting the paired sample t- test, the normality assumptions were checked for Pre and Post-test scores. The results showed that skewness was .07 and kurtosis was .72 based on the math growth of the participants. Considering skewness and kurtosis together, the results are close to meeting the normality requirement just outside the acceptable range.

The test resulted in a significant difference between the participants pre-test math score (M = 200.86, SD = 11.47) and post-test math score (M = 194.38, SD = 15.68). Table 3 T-test Statistics Comparing Student Growth in Math for Summer Academy Participants n Mean SD SE Pre-Test Math Score 56 200.86 11.47 1.51 Post-Test Math Score 56 194.38 15.68 2.06

The participants significantly regressed in math even though they participated in Summer Academy, t(57) = 5.00, p < .05. The Cohen’s d has a value of .65, which is medium. There is a medium impact regarding math growth for participants in Summer Academy. Table 4 Paired Sample T-test Results Comparing Student Growth in Math for Summer Academy Participants Mean SD SE t df Sig. Pre-Test Math Score - Post-Test Math Score -6.48 9.95 1.31 5.00 57 .00 Cohen’s d=0.65 Research Question 2A: How do the reading growth scores between Summer Academy participants and non-participants compare?

In order to examine whether Summer Academy non-participants demonstrated growth in reading, the researcher conducted a paired sample t-test. Before conducting the paired sample

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

59

t-test, the normality assumptions were checked for pre and post-test scores. The results showed that skewness was .44 on the pre-test and .36 on the post-test. The value of kurtosis was 1.08 on the pre-test and .97 on the post-test. Considering skewness and kurtosis together the results meet the normality assumption.

The test resulted in a significant difference between the non-participants pre-test reading score (M = 212.67, SD = 12.80) and post-test reading score (M = 216.47, SD = 13.47). Table 5 T-test Statistics Comparing Student Growth in Reading for Summer Academy Non-Participants n Mean SD SE Pre-Test Reading Score 357 212.67 12.80 0.68 Post-Test Reading Score 357 216.47 13.47 0.71

The non-participants did not demonstrate as much growth in reading between the pre and post-test as those students that participated in Summer Academy, t(358) = 9.57, p < .05. The Cohen’s was 0.50 and although students maintained literacy skills through independent reading and other experiences, students did not show the same growth as those that participated. Table 6 Paired Sample T-test Results Comparing Student Growth in Reading for Summer Academy Non- Participants Mean SD SE t df Sig. Pre-Test Reading Score - Post-Test Reading Score 3.80 7.53 0.40 9.57 358 .00 Cohen’s d=0.50

When comparing the growth of the Summer Academy participants with the Summer Academy non-participants, the effect size of the participants is large whereas the non- participants is medium. The mean growth is more than double in the participant group. More work still needs to be done as the mean score in this group is still below grade level. Table 7 Independent Sample T-test Comparing Student Growth in Reading for Summer Academy Participants and Non-Participants Levene's Test for Equality of Variances t-test for Equality of Means Sig. 95% Confidence MD Std. Error Interval of the Difference F Sig. t df p Difference Lower Upper Eq. Var. 7.127 .008 -3.558 411 .00 -3.96 1.11 -6.15 -1.77 Not Eq. Var. -3.041 66.65 .00 -3.96 1.30 -6.56 -1.36

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

60

Research Question 2B: How do the math growth scores between Summer Academy participants and non-participants compare?

In order to examine whether Summer Academy non-participants demonstrated growth in math, the researcher conducted a paired sample t-test. Before conducting the paired sample t-test, the normality assumptions were checked for pre and post-test scores. The results showed that skewness was .01 on the pre-test and .02 on the post-test. The value of kurtosis was .12 on the pre-test and .24 on the post-test. Considering skewness and kurtosis together, the results meet the normality assumption.

The test resulted in a significant difference between the non-participants pre-test math score (M = 220.45, SD = 14.52) and post-test math score (M = 217.14, SD = 18.12). Table 8 T-test Statistics Comparing Student Growth in Math for Summer Academy Non-Participants n Mean SD SE Pre-Test Math Score 357 220.45 14.52 0.77 Post-Test Math Score 357 217.14 18.12 0.96

The non-participants significantly regressed in math between the pre and post-test, t(358) = 4.86, p < .05. The Cohen’s d was .26. There is little to no impact regarding math growth for non- participants in Summer Academy. Table 9 Paired Sample T-test Results Comparing Student Growth in Math for Summer Academy Non- Participants Mean SD SE t df Sig. Pre-Test Math Score - Post-Test Math Score -3.32 12.95 0.68 4.86 358 .00 Cohen’s d=0.26 Table 10 Independent Sample T-test Comparing Student Growth in Math for Summer Academy Participants and Non-Participants Levene's Test for Equality of Variances t-test for Equality of Means Sig. 95% Confidence MD Std. Error Interval of the Difference F Sig. t df p Difference Lower Upper Eq. Var. 10.93 .00 3.24 411 .00 3.34 1.03 1.31 5.36 Not Eq. Var. 2.46 63.18 .02 3.34 1.36 .62 6.05

When comparing the regression of the Summer Academy participants with the Summer

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

61

Academy non-participants, unfortunately the effect size of the participants is medium whereas the non-participants is small. The mean regression is almost double in the participant group.

The findings of this research indicate that the students that participated in Summer Academy grew in reading almost double compared to those students that did not participate in Summer Academy. Students that participate in Summer Academy score in the bottom 25%. According the nationally normed data. The growth of these students is the most important as they are well below grade level. Summer Academy is designed to remediate reading and writing skills over the course of 5 weeks. Although this is positive, the mean score on the post-test is still below grade level standard and nearly 22 points below the non-participants, which is example that the achievement gap exists. More work needs to be done throughout the year and summer to close this gap.

The findings of this research also indicate that the students that participated in Summer Academy regressed in math almost double compared to those students that did not participate in Summer Academy. Students that participate in Summer Academy score in the bottom 25%. According the nationally normed data. The growth of these students is the most important as they are well below grade level. Summer Academy is designed to remediate important skills over the course of 5 weeks. The scores are disheartening, as the mean score on the post-test is still below grade level for all students and the achievement gap is widening. More work needs to be done to enhance the math component of Summer Academy as well as the math requirement of all students during the summer months. Summer Academy does not have a math component, so the participants, which are students with the highest needs, regressed more than the non- participants because of no math instruction during the summer months. This demonstrates a need for a math component of summer academy as well as a more impactful summer math assignment for all students. The implications for policy and practice will be discussed in the next chapter.

Discussion

Research Question 1: What is the impact of the Summer Academy on students’ achievement? Students were recommended for Summer Academy based on their reading scores from the NWEA map test, and the focus of the program was around literacy; therefore, alternative hypothesis 1a is supported. The NWEA map test is a research based, norm-referenced test administered to students in a group setting in September and April of each school year. Students take the assessment on the computer; it is an online, adaptive benchmark and progress monitoring assessment that efficiently measures oral reading fluency, literal comprehension, and foundational skills in math and reading. The reading and math tests are separate and include a combination of multiple choice and open-ended questions. The test is untimed, but the average student completes it in about 90 minutes.

The focus of Summer Academy was reading and literacy as seen in the curriculum as well as lesson design and resources. The curriculum and daily instruction focused primarily on reading and literacy. The schedule included reading as a whole group, reading in smaller groups, independent reading, writing, and science-bases performance tasks. Because of 80% of the students throughout Summer Academy specifically working on reading and general literacy,

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

62

comparing the average pre-test reading score with the average post-test reading score of all participants shows a growth of over eight points.

The study from Roman and Fiore (2010) shows students who participate in a summer reading program score higher on the reading achievement test at the beginning of specific grade levels and do not experience summer reading loss. Roman and Fiore’s study supported the findings of previous research studies in the following ways: students who participated in the summer reading program maintained and increased their reading skills, recreational reading outside of school made a difference in improving reading scores, and libraries being accessible to all students is essential regardless of economic status (Roman & Fiore, 2010). This is an area of improvement as noted in recommendations.

Students were recommended for Summer Academy based on their reading scores as opposed to math and the focus of the program was around literacy; therefore, the alternative hypothesis 1b is accepted. Students were not recommended for Summer Academy based on their math scores and there was no math instruction during that time. As a result, Summer Academy participants regressed in math between their pre-test in math and their post-test in math. These are the students with the greatest need. The program was effective with regard to increasing literacy skills for participants, but not math. This is an area in need of improvement as noted in the recommendations.

In a different study regarding math summer learning loss, Slates, Alexander, Entwisle, and Olson (2012) found that learning loss was more pronounced in the area of math than any other content area or topic. Moore (2010) explained that struggling math students typically participate in fewer activities at home during the summer months and therefore experience more of a loss than other students experience. Research Question 2: What is the academic growth between Summer Academy participants and non-participants? Summer Academy participants grew more than non-participants in reading based on a comparison between individual pre-test and post-test scores, which supports alternative hypothesis 1a. The non-participants did not grow enough to close the achievement gap; however, students that participated scored well below grade level on the post-test even after Summer Academy. Participants were selected for Summer Academy because of their scores in reading and all were well-below grade level.

Non-participants’ pre-test reading scores and their post-test reading scores were higher than those of participants. These results are consistent with the research. The National Assessment of Educational Progress (NAEP) results indicate that recent efforts to increase the percentage of students scoring above the basic level have not resulted in large improvements (National Center for Education Statistics, 2014).

Prior research indicates that reading comprehension proficiency arises through an interaction between exposure to text (Pucell-Gates, Jacobsen, & Degener, 2004), self-efficacy beliefs about reading competence (Bandura, 1977), motivation to read voluntarily (Wigfield Eccles, & Rodriquez, 1997), and reading practice (Heyns, 1978). These studies demonstrate that students that show these characteristics before the assessment would be more likely to achieve

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

63

higher than those students that did not. Increased access to books such as the school or public library would provide that opportunity which is addressed in the recommendations. Both participants and non-participants regressed in math from their pre-test to their post- test, which supports the alternative hypothesis 2b. Non-participants had a higher pre-test and post-test math score. Because of their proficiency level and access to enrichment over the summer, they were able to maintain more skills than their peers that participated in Summer Academy. Again, the participants were scoring below grade level in reading, which affects their ability to be successful in some components of math such as word problems.

There was no math instruction during Summer Academy. There was an opportunity, but the instructional time was focused on science and students engaging in performance tasks. Linder (2010) emphasized this as a necessary strategy for students who struggle with mathematics and explained that completing an alignment process between the textbook and the math standards helped teachers focus more on the individual student, design specific strategies for connecting students to a lesson, and help them experience success in each lesson. This did not occur and is addressed in the recommendations.

One previous study from the NAEP showed little improvement between 2007 and 2013 in the percentage of students scoring at the proficient level or above (National Center for Education Statistics, 2014) in math. Although school districts have provided extra math instruction during the regular school day, through tutoring after school, and through summer school programs, math achievement still has not increased (Krawec, et. al, 2013). The necessity of direct math instruction is evident. General Implications for School Leaders The need for summer school programs is clearly defined in the research, which indicates that students experience approximately one month’s worth of academic loss in math and literacy during the three months of summer vacation (Cooper, 2003). However, specific strategies are suggested to not only mitigate this summer loss but to also show academic gains. These strategies include small class sizes, individualized instruction, cooperative learning, rewards, and standards- based report cards (Jesson et al., 2014). Because of the shorter time frame available during a summer program, the following should occur at minimum: complex topics should be taught early in the program, assignments should be shortened to an appropriate length, and a minimum of 30 hours of instruction should be delivered over the course of the summer program in order for students to experience an increase in literacy and math achievement (Zvoch & Stevens, 2013).

The importance of this research is that for the first time, it emphasizes the effect school leaders have in designing research-based summer programs that are aligned to the regular school year (McCombs, 2011). School leadership development is integral to the success of summer programs especially with regard to the impact programs have on economically disadvantaged students. The research has been clear on the elements to close the achievement gap, but what is missing is the mechanism that needs to be in place and how the educational leaders need to impact all aspects of it. This includes instructional strategies, time, location, and attendance expectations for students.

Studies that analyze educational leaders of summer programs in other geographic locations are crucial to support the general improvement. Using the same survey across different

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

64

geographical regions and school sizes may help to make clear what hinders or supports schools in creating comprehensive summer learning environments. The inclusion of those principals’ voices would also help to clarify what obstacles stand in the way of principals, creating more opportunities in this environment.

Educational leaders also must create a three-year strategic plan that will outline goals, identify expected outcomes, and name potential funders. With this data accumulated from a first year summer program, educational leaders need to consider the successful areas and how they can act as a foundation for school change. These are all areas to be considered as when planning programming during the following schoolyear.

Parents and community partnerships are assets that can support and grow sustainable summer programs. Educational leaders must create a plan to identify community assets, build relationships with these extended community stakeholders, and implement a plan of action to benefit the summer program with community resources. These assets decrease programming costs, assist personnel, and garner additional funding from outside resources (McCombs, 2011). Parents and community partners want meaningful relationships with schools. Educational leaders can develop them by creating a community engagement plan that identifies measurable goals and assets, timelines, and ways to sustain relationships. Planning for sustainability remains difficult because so many current factors need attention, thereby interfering with considering future needs (McCombs, 2011).

Without effective leadership, the chances for systemic improvement in teaching and learning are futile (Tirozzi, 2001). Given the findings from numerous studies that have found positive relationships between principals’ practices and various school outcomes, policymakers and educational experts are increasingly turning to educational leadership development as a strategy for improving schools and student achievement (Orr & Orphanos, 2011). Implications for Social Justice and Equity The problem with math and literacy achievement is both a national and a local problem. The longitudinal results from the NAEP showed that there has been little if any progress with math achievement at the fourth and eighth grade levels (National Center for Education Statistics, 2014). If a summer math or literacy program could be designed to effectively address the problem of achievement at the local level, it would have a great impact for positive social change within the local district. Diversity Middle School needs to move in this direction by incorporating a math curriculum into Summer Academy that aligns with the curriculum used during the regular school year, as well as making the literacy curriculum more robust. The school can make adjustments that will lead to social change by identifying aspects of the program that can be modified to increase its effectiveness. As the school district improves its math and literacy instructional programs, students will be better prepared for high school and beyond. Proficiency in math and literacy will benefit students whether they enroll in a college or university or enter the workforce. An effective summer school program has the potential to generate a great deal of social change for the students involved and for the local community.

How moral principles are related to social justice is apparent in Glickman and others’ (2009) example of how inclusion combines the beliefs in equality and equity. Summer programming provides opportunity for learning to continue for all students. As identified,

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

65

students that receive special education services or have ELL status need more support than those students that do not. All students are of equal worth as human beings and as members of the school community. A belief in that moral principle maintains a commitment to equity by providing special assistance to those with specific needs to enable them to remain members of the community and lead fulfilling lives as students and later as adults. Glickman et al. added that a good school actually reaches out to all categories of students. This is how an effective summer program can not only close the achievement gap for its students with the highest needs, but also create systemic change as those students achieve and have a more successful future.

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

66

References

Augustine, C. H., McCombs, J. S., Schwartz, H. L., & Zakaras, L. (2013). Getting to work on summer learning: Recommended practices for success(Rep.). Santa Monica, CA: RAND Corporation.

Alexander, K. L., Entwisle, D. R., & Olson, L. S. (2007). Lasting consequences of the summer learning gap. American Sociological Review,72(2), 167-180. doi:10.1177/000312240707200202

Allington, R. L., & McGill-Franzen, A. (2003, November). Use students’ summer setback months to raise minority achievement. Education Digest,69(3), 19-24.

Allington, R. L., McGill-Franzen, A., Camilli, G., Williams, L., Graff, J., Zeig, J., Zmach, C., & Nowak, R. (2010). Addressing summer reading setback among economically disadvantaged elementary students. Reading Psychology ,31(5), 411-427. doi:10.1080/02702711.2010.505165

Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191-215.

Bowling A., Ebrahim S. (2005) Handbook of health research methods: Investigation, Measurement and Analysis. Open University Press Maidenhead.

Chan, D.W. (2000). Dimensionality of hardiness and its role in the stress-distress relationships among Chinese adolescents in Hong Kong. Journal of Youth and adolescence, 29(2), 147- 161.

Christiansen, J., Christiansen, J. L., & Howard, M. (1997). Using protective factors to enhance resilience and school success for at-risk students. Intervention in School and Clinic, 33(2), 86-89.

Cooper, H. (2003). Summer learning loss: The problem and some solutions. ERIC Clearinghouse on Elementary and Early Childhood Education. Retrieved from: http://www.ericdigests.org/2003-5/summer.htm

Corcoran, J., & Nichols-Casebolt, A. (2004). Risk and resilience ecological framework for assessment and goal formulation. Child and adolescent social work journal, 21(3), 211- 235.

Creswell, J. W. (2017). Educational research: planning, conducting, and evaluating quantitative and qualitative research (4th ed.). Boston: Pearson.

Cronin, J., Dahlin, M., Adkins, D. & Kingsbury, G. G. (2007). The proficiency illusion. Portland,OR: Northwest Evaluation Association.

Fifer, M. E., & Krueger, A. B. (2006). The Hamilton Project: Advancing opportunity, prosperity and growth. PolicyBrief, 3, 1-8.

Fredrickson, B.L. (2001). The role of positive emotions in positive psychology: The broaden-and- build theory of positive emotions. American Psychologist, 56(3), 218-226.

Glickman, C.D., Gordon, S.P. and Ross-Gordon, J.M. (2009), The Basic Guide to Supervision and Instructional Leadership, 2nd ed., Pearson Education-Allyn & Bacon, Boston, MA.

Hanover Research (2017). Best practices for comprehensive summer school programs. (pp. 1- 31, Rep.). Arlington, VA: Hanover Research.

Heyns, B. (1978). Summer learning and the effects of schooling. New York: Academic Press.

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

67

Keiler, L. S. (2011). An effective urban summer school: Students’ perspectives on their success. Urban Review, 43(3), 358–378. https://doi.org/10.1007/s11256-010-0154

Kotter, J. P. (2012). Leading change. Boston, MA: Harvard Business Review Press. Krawec, J., Huang, J., Montague, M., Kressler, B., & de Alba, A. M. (2013). The effects of

cognitive strategy instruction on knowledge of math problem-solving processes of middle school students with learning disabilities. Learning Disability Quarterly, 36(2), 80- 92. doi:10.1177/0731948712463368

Jesson, R., McNaughton, S., & Kolose, T. (2014). Investigating the summer learning effect in low SES schools. Australian Journal of Language and Literacy, 37(1), 45. Retrieved from http://www.alea.edu.au/publications

Kuhfeld, M. (2019). Surprising new evidence on summer learning loss. Phi Delta Kappan, 101(1), 25–29. https://doi.org/10.1177/0031721719871560

McCombs, J. S., Augustine, C. H., Schwartz, H. L., Bodilly, S. J., McInnis, B., Lichter, D. S., & Cross, A. B. (2011). Making summer count: How summer programs can boost children’s

learning. Santa Monica, CA: Rand Corporation. McEachin, A., Augustine, C., & McCombs, J. (2016). Best practices in summer programming. In

Alexander, K. L., Pitcock, S., & Boulay, M. (Eds.) The summer slide: what we know and can do about summer learning loss (pp. 193-207). New York: Teachers College Press.

Moore, C. (2010). The effects of summer vacation on mathematical knowledge of rural students transitioning from third to fourth grade. Journal for the Liberal Arts and Sciences, 14(2), 58. Retrieved from http:// http://www.oak.edu/academics/schoolarts-sciences-jlas.php

Mraz, M.,& Rasinski, T. (2007). Issues and trends in literacy, summer reading loss. The Reading Teacher,60(8), 784-789.

Murray, C. (2003). Risk Factors, Protective Factors, Vulnerability, and Resilience: A Framework for Understanding and Supporting the Adult Transitions of Youth with High-Incidence Disabilities. Remedial and Special Education, 24(1), 16– 26. https://doi.org/10.1177/074193250302400102

National Center for Education Statistics (2014). The Nation's report card: 2013 Mathematics and Reading Grade 12 Assessments. National Center for Education Statistics. Retrieved from https://nces.ed.gov

Orr, M. T., & Orphanos, S. (2011). How graduate-level preparation influences the effectiveness of school leaders: A comparison of the outcomes of exemplary and conventional leadership preparation programs for principals. Educational Administration Quarterly, 47(1), 18-70.

Öz, H. & Özturan, T. (2018). Computer-based and paper-based testing: Does the test administration mode influence the reliability and validity of achievement tests?. Journal of Language and Linguistic Studies, JLLS Mart 2018 Sayısı, 67-85. Retrieved from https://dergipark.org.tr/en/pub/jlls/issue/43213/527697

Pedditzi, M. L., Fadda, R., & Lucarelli, L. (2022). Risk and Protective Factors Associated with Student Distress and School Dropout: A Comparison between the Perspectives of Preadolescents, Parents, and Teachers. International Journal of Environmental Research and Public Health, 19(19), 12589. https://doi.org/10.3390/ijerph191912589

Purcell-Gates, V., Jacobson, E., & Degener, S. (2004). Print Literacy Development: Uniting Cognitive and Social Practice Theories. Cambridge, MA: Harvard University Press.

Education Leadership Review of Doctoral Research, Volume 10, Fall 2022

68

Roman, S., & Fiore, C. D. (2010). Do public library summer reading programs close the achievement gap? The Journal of the Association for Library Service to Children, 8(3), 27-3.

Rutter, M. (1987). Psychosocial resilience and protective mechanisms. American Journal of Orthopsychiatry, 57(3), 316- 331.

Schien, E. H. (2010). Organizational culture and leadership (4th ed.). San Francisco: JosseyBass. Slates, S. L., Alexander, K. L., Entwisle, D. R., & Olson, L. S. (2012). Counteracting summer slide:

Social capital resources within socioeconomically disadvantaged families. Journal of Education for Students Placed at Risk, 17(3), 165-185. doi:10.1080/10824669.2012.688171

Smith, L. (2012). Slowing the summer slide. Educational Leadership, 69(4), 60-63. Tirozzi, G. N. (2001). The artistry of leadership: The evolving role of the secondary school

principal. Phi Delta Kappan, 82(6), 434-439. Travis, Raphael, Jr., Gann, Elliot, Crooke, Alexander H.D., & Jenkins, Susan M. (2019) Hip Hop,

empowerment, and therapeutic beat-making: Potential solutions for summer learning loss, depression, and anxiety in youth, Journal of Human Behavior in the Social Environment, 29:6, 744-765, DOI: 10.1080/10911359.2019.1607646

Ungar, M. (Ed.). (2005). Handbook for working with children and youth Wigfield, A., Eccles, J. S., & Rodriquez, D. (1998). The development of children’s motivation in

school contexts. Review of Research in Education, 23, 73-118. Zvoch, K., & Stevens, J. J. (2011). Summer school and summer learning: An examination of the

short-and longer term changes in student literacy. Early Education & Development, 22(4), 649-675. doi:10.1080/10409289.2010.489891