Edducation funding proposal

NIH Proposal

The Effects of Special Education Preschool on Non-Academic Outcomes Research Questions and Hypotheses: Question 1: What are the effects attending special education preschool (SEP) on children’s school readiness skills and their transition to Kindergarten? Question 2: What are the effects attending SEP on children’s non-academic outcomes at the end of Kindergarten? Question 3: What are the effects attending SEP on children’s non-academic outcomes at the end of third grade? Question 4: Do the effects of SEP depend on a child’s disability type or where they received their special education preschool services? H1: The evidence on the effects of SEP is mixed. Past literature on the effectiveness of SEP has found negative effects on math and reading scores at Kindergarten entry (Sullivan & Field, 2013). In a national sample of children who attended SEP, Carson et al. (2008) found increases in some academic, social, and motor skills after one year. However, there were no comparisons to peers who did not attend. About 38% of children who enter PSE leave within the normal range on five developmental delay measures, but this percentage varies depending on the extent of a child’s delay when they enter (Elbaum, 2020). Additionally, low-income children who received SEP have been found to score significantly lower on a variety of school readiness skills than children who did not. While some variables were controlled for, selection bias was not fully accounted for. Further, this analysis was not the primary purpose of the data set, so other confounding variables may be present (Joen et al., 2011). Since our analysis will take into account selection effects, we hypothesize that there will be either negative or null findings on school readiness. We have not found any studies about SEP and adult perceptions of kindergarten transition; thus, we cannot predict a direction for the effect. H2: There has not been much prior research on the effects of SEP at the end of Kindergarten. Given the null or negative findings for SEP (Sullivan & Field, 2013) and regular special education (Morgan et al., 2010), we hypothesize that there will either be null or negative effects. H3: Research is mixed on whether or not positive outcomes from high quality preschool persist until third grade or beyond (Barnett, 2011; see Winsler & Mumma, 2021 for a review), but research has yet to investigate the persistence of non-academic effects from SEP to elementary school. Studies have failed to find long-term academic effects of SEP (Jenkins et al., 2006) or short-term (e.g., Morgan et al., 2010) and long-term (Kanaya et al. (2019) academic and non-academic effects of special education. Thus, it is likely that we will find null or negative results of SEP. H4: Children who receive SEP are diagnosed with many different disabilities and even children with the same diagnosis can have different strengths and challenges. Research has shown that the extent of a child’s delay at the end of SEP varies based on the complexity and severity of the delay (Elbaum, 2020). Because of this, we believe that the effectiveness of SEP will vary depending on a child’s disability diagnosis. Secondly, research suggests that there are differences in both structural and process quality across preschool type (Coley et al., 2016) and that children can have different outcomes based on what preschool type they attend (e.g., Coley et al., 2016; Henry et al., 2006). Thus, we believe that preschool type will moderate the effectiveness of SEP. Primary Source of Data: Early Childhood School Readiness Survey(ECSRS)

The Early Childhood School Readiness (ECSRS) data set is a (fictious) nationally representative data set containing information on a cohort of students in the U.S. who were in Kindergarten in the 2012-2013 academic year and follows them until third grade (2016-2017). While data is collected in the

NIH Proposal

fall and spring of each grade, for this study, we are only focused on data collected in the fall and spring of the Kindergarten year and in the spring of third grade. Similar to other nationally representative surveys (like the ECLS-K), data is collected directly from students, their parents, and educators (classroom teachers, special education teachers, and early care and education teachers). What is unique about this data set, when compared to other nationally representative data sets, is that it over-samples children who attend SEP and those at risk of being placed in special education when they begin school. This allows for a variety of analyses related to SEP and elementary school special education that are not possible in other data sets that have much smaller sample of students with disabilities. Participants. Our analytical sample consists of 10,000 children. Of that 10,000, 50% are students without disabilities. The other 5,000 receive special education services in Kindergarten, and of that, 2,500 children attended SEP. Similar to national statistics, about 60% of students in the sample are White, 12.5% are black, 18.5% are Hispanic, 5.8% are Asian, and the remainder are either Pacific Islander, American Indian/Alaskan Native, or other (NCES, 2020). Kindergarten Readiness Measures. School readiness is a complex concept that is thought of in three different layers: the child, the family, and community/society (Kagan et al., 1995). While all three levels are important, this analysis will focus in on the child. A child’s school readiness is often thought of as being made up of five dimensions: physical well-being/motor development, social and emotional development, approaches towards learning, language development, and cognition/general knowledge (Kagan et al., 1995). All students in this sample were assessed on the domains prior to school starting. Physical Well-Being/Motor Development: This information is collected by an occupational therapist trained in the CHAMPS Motor Skills Protocol (CMSP) (Williams et al., 2008). The CMSP consists of six locomotor (run, jump, slide, gallop, leap, hop) and six object control (throw, roll, kick, catch, strike, dribble) behaviors. Each part of the movement to perform the behavior is given a score of 0 (absent) or 1 (present). Composite scores are calculated for each type of behavior and a total. Scores range from 0-73 for locomotor skills, 0-80 for object control, and 0-153 for total (Williams et al., 2008). Information about children’s current physical well-being (like height, weight, and regular physical activity) are collected from parents. Social and Emotional Development: Similar to the ECLS-K, social and emotional development is measured using both teacher and parent questionnaires. Teachers rate the child’s attentional focus and inhibitory control using scales from the Children’s Behavior Questionnaire and self-control, interpersonal skills, externalizing problem behaviors and internalizing problem behaviors from a modified version of the Social Skills Rating System (Najarian et al., 2018). The Children’s Behavior Checklist is scored on a scale from 1 (extremely untrue) to 7 (extremely true). Teachers can also answer “not applicable” if they have not witnessed a child display that behavior. The Social Skills Rating System is measured on a scale from 1 (never) to 4 (very often) and teachers also can indicate if they have not seen the child display those particular behaviors (Najarian et al., 2018). Parents also rate their child’s social and emotional skill using items from the Social Skills Rating System. The constructs on the parent measure are self-control, social interaction, sad/lonely, and impulsive/overactive behaviors (Najarian et al., 2018). Approaches Towards Learning: Teacher-rated approaches to learning is measured using the seven items used on the ECLS-K. It is rated on a scale from 1 (never) to 4 (very often) and teachers can also rate “not applicable” (Najarian et al., 2018). Parents also rate their child’s Approaches to Learning using a similar scale as teachers (Najarian et al., 2018). Language Development: Language Development consists of both a child’s verbal language and emerging literacy skills (Kagan et al., 1995). The ECSRS has its own measure to assess both of these skills in children. The verbal language measure tests a child’s listening, speaking, social issues of

NIH Proposal

language, vocabulary and meaning, questioning, and creative uses of language skills (Kagan et al., 1995). The emerging literacy measure assesses children’s literature awareness, print awareness, story sense, and writing process (Kagan et al., 1995). Scores are summed to create an overall language development score. Cognition/General Knowledge: Cognition and general knowledge consists of three domains: physical, logico-mathematical, and social-conventional knowledge (Kagan et al., 1995). ECERS created tools to assess learning in each domain. The scores are summed to create an overall cognition/general knowledge score. Children’s Transition to Kindergarten: Both parents and teachers complete a survey regarding how ready the child was for Kindergarten. The 13 kindergarten readiness items are taken from the teacher questionnaire on the ECLS-K (NCES, n.d.). Instead of rating how important the skills are for kindergarten entry, both parents and teachers rate the child’s abilities on a scale from 1 (not present) to 4 (advanced). End of Kindergarten Measures. All school readiness measures were taken again at the end of the Kindergarten year.

Classroom Engagement. Children’s classroom engagement was measured at the end of Kindergarten using the Classroom Engagement Scale (Barghaus et al., 2017). This 14-item scale assesses children’s academic and social engagement. Skills are rated on a 3-point scale: 0 (Improvement Needed), 1 (Satisfactory), and 2 (Outstanding). Third Grade Outcomes. The same assessments of social-emotional development, Approaches to Learning, and Classroom Engagement that were given to students in Kindergarten were given again in third grade. If necessary, updates to items on all assessments were made to make them more developmentally appropriate. SEP Attendance. The ECSRS asks both parents and the child’s special education teacher if the child attended SEP (or received services at their home). Under the Individuals with Disabilities Education Act (IDEA) children with disabilities between the ages of 3 and 5 are entitled to special education services which they can receive in their preschool, child care, or Head Start program. If a child is not in one of these settings, schools are required to provide services to the child in another setting (i.e., their home) (NCLD, n.d.). If either the parent or special education teacher indicated that the child attended SEP, it was coded as such. Special Education Status. The fall parent and special education teacher questionnaire also asks if the child currently has an IEP and is receiving special education services in Kindergarten. If either the parent or special education teacher indicated the child has an IEP, it was coded as such. Covariates. The ECERS data set contains a rich amount of data on child-level, family-level, and school-level characteristics. This is important in order to for us to be able to accurately predict the likelihood of a student attending SEP or not. Past research looking at the effectiveness of special education has included a variety of covariates including gender, race, socio-economic status, maternal age, and child care experiences prior to Kindergarten (e.g., Morgan et al., 2010; Sullivan & Field, 2013). Variables that have either empirical or theoretical support to predict a child attending SEP that are in our data set are included in the analysis. Analysis: Question 1: What are the effects attending SEP on children’s transition to Kindergarten?

To answer this research question, we propose a propensity score matching (PSM) analysis. PSM is used to control for selection bias when you have pre-existing groups (in our case children who attended SEP and those who did not) (Class Notes). Using all of the covariates that are empirically or

NIH Proposal

theoretically thought to be related to special education identification, for each child in the sample, we will create propensity scores representing the likelihood that they would be enrolled in SEP (Class Notes). As in Morgan et al., 2010, we will use multiple matching techniques (i.e., kernel matching, nearest neighbor, and stratification matching).

Once each individual is given a propensity score, we will create “blocks” of individuals in both the treatment (SEP) and control (non-SEP) groups based on their score. The creation of blocks is an iterative process. First, we will need to ensure that there are both treatment and control students in each block. Next, we look at the average propensity scores for each group in every block to make sure they are equal. Then, we compare the average covariate values in both groups in every block to be sure that they are the same. If any of these conditions are not met, we need to create new blocks until they are all met (Class Notes). Once our propensity scores have been created and have met all the necessary conditions, we can move ahead with analysis. For each block, we will estimate the treatment effect. We then calculate a weighted average to obtain an estimate of the overall effect. This is called the weighted average treatment effect (Class Notes). This will be done separately for each outcome of interest.

For an additional comparison, we will create propensity scores and conduct the analysis twice: once with the “control” group being all non-SEP students and the other being students who were classified as special education in kindergarten, but did not attend SEP. This will help us understand if the effects of attending SEP are different when students who received it are compared to students who potentially could’ve received it but were not identified prior to Kindergarten than when they are compared to all non-SEP students. For both sets of analyses within-block comparisons will be run (like in Morgan et al., 2010). This will tell us if the effectiveness of SEP depends on a child’s likelihood of attending.

Question 2: What are the effects of SEP on student’s non-academic outcomes at the end of Kindergarten?

We will use the propensity scores that were created in Question 1 for this analysis. For outcomes that we have at two-time points (beginning and end of Kindergarten), we will estimate both the end of year scores and the gain score. For measures that we only have at the end of Kindergarten, we will only be able to estimate the spring score. Question 3: What are the effects of SEP on student’s non-academic outcomes at the end of third grade?

Similar to Question 2, we will use the propensity scores from Question 1. Also like Question 2, we will estimate the effects for scores at the end of third grade and the gain score for measures we also collected in Kindergarten. The gain score will be calculated from fall of Kindergarten to third grade. Question 4: Do the effects of attending SEP to Kindergarten depend on a child’s disability type or where they received their special education preschool services?

In order to test the moderating effects of disability and preschool type on the effectiveness of SEP, we will need to add dummy variables into a regression equation with SEP as the predictor and the outcomes of interest as the dependent variable. These dummy variables will represent the different types of disabilities and preschool settings. This will be done for all three questions mentioned above, as it possible that differences exist for some outcomes at certain grades and not all of them.