Project 3 FEA Plan

FEA PLAN FOR INCREASING STUDENT ENGAGEMENT 24

Front-End Analysis Plan for Increasing Student Engagement

at Penn Wood High School, Lansdowne, PA

Leigh M. Tolley

Syracuse University

May 1, 2010

Submitted to:

Jing Lei, Ph.D.

Course Instructor

Running head: FEA PLAN FOR INCREASING STUDENT ENGAGEMENT 1

EVALUATION OF SUNY CORTLAND’S C.A.S. SDL PROGRAM 5

IDE 712 — Analysis for Human Performance Technology Decisions

Identifying the Performance Problem

Student engagement at Penn Wood High School (PWHS) in Lansdowne, PA is currently very low. It is not uncommon to observe students not paying attention in class or roaming the hallways, and daily attendance is at about 80-85%, which includes students who arrived at school an hour or more late. Students frequently complain that classes are boring, too difficult, or both, and a disconnect exists between students and many of the staff. Inappropriate behavior is common at the school, with many office referrals written daily, an ever-present in-school suspension system, and security guards tasked with making sure students are in class, as well as stopping the occasional fight. Students say that they want to learn, but do not believe that they have the skills necessary to succeed. Despite many positive programs that help students to achieve, including several strong Advanced Placement courses, the school has a reputation for not having high academic standards, both among the students and in the community. Students at the school often perform poorly on standardized tests, despite many intervention efforts over the last ten years. Although all of these factors are troubling on their own, together they comprise the construct of student engagement, which is severely lacking at PWHS. Comment by Jing Lei: I’m glad you finally were able to concretely state the problem.

Context of the Performance Problem

PWHS is a grade 10-12 urban high school in the southwest suburbs of Philadelphia. The school has over 1200 students (predominantly African-American) and about 75 teachers. About 40 percent of the students in 2007-2008 were eligible for free or reduced-priced lunches (National Center for Education Statistics). Approximately one-third of the student population has been identified as having special needs, requiring an individualized education plan (IEP), or a 504 Plan for accommodations, and a small number of students are identified as gifted and talented. The school also has about 30 English Language Learner (ELL) students.

In 2008-2009, 385 of the school’s 11th graders tested by the Pennsylvania System of School Assessment examinations (PSSAs) scored the following proficiencies in math: 7.0% Advanced; 22.6% Proficient; 21.8% Basic; 48.6% Below Basic. In that same testing period, 383 11th graders tested by the PSSAs scored the following proficiencies in reading: 8.9% Advanced; 22.2% Proficient; 23.2% Basic; 45.7% Below Basic (Pennsylvania Department of Education). The school’s Adequate Yearly Progress (AYP) status in 2009 was in its third year of Corrective Action 2 (Pennsylvania Department of Education). This is the lowest level of AYP status under No Child Left Behind (NCLB), and leaves the school subject to governance changes such as reconstitution, chartering, and privatization (AYP Status Levels). The PSSA scores are well below those needed for AYP: at least 56% of the tested students must achieve a Proficient score or higher on the mathematics assessment, and 63% of the tested students must achieve a Proficient score or higher on the reading assessment (AYP Facts).

Since 2003, the school has not met AYP, and consecutively received a warning, one year in School Improvement 1, one year in School Improvement 2, a year in Corrective Action 1, and three years in Corrective Action 2 (AYP Status Levels). This steady decline in performance, assessed mainly through PSSA results, has led the school to enact several interventions, including implementing America’s Choice programs for reading and math, the revision of district-wide curricula to meet state standards, and the instituting of quarterly benchmark testing for students. The school culture has become test-driven in order to try to meet AYP, but student engagement has suffered as a result. Decreased student engagement has continued to affect PSSA test results, as students are more reluctant to attend school or pay attention while there if the focus is on standardized testing.

A very clear introduction of the context!

Continuing to neglect the importance of student engagement would allow the school to continue in Corrective Action, perhaps even to the point of it being taken over by the state. An analysis of the problem of student engagement and its relationship to performance on the PSSAs, as well as overall student outcomes, needs to be conducted. The problem as it currently exists affects students, teachers, administrators, the school, and the district as a whole, as PWHS is the solitary high school in the district. The students at PWHS are currently at a low level of engagement; they need to be at a level of engagement where they are successful socially and academically, and where they have the skills, knowledge, and persistence to earn their high school diplomas and continue to succeed after graduation.

Clarifying the Construct of Student Engagement through a Literature Review

In order to learn more about student engagement, its definition, its benefits, how it can be measured, and how interventions can help increase it, a literature review was conducted. What follows is a summary of this review, including key questions that were asked in order to clarify the construct of student engagement.

Why Is Student Engagement Important? Comment by Jing Lei: Importance well justified! And I like how you used research to support your arguments.

Previous research has indicated a positive relationship between students’ engagement and their grades and scores on standardized tests (Caraway & Tucker, 2003; McClenney, 2007; Weiss, Carolan, & Baker-Smith, 2010). Strong academic or school engagement is associated with higher academic and developmental outcomes (Appleton, Christenson, & Furlong, 2008; Baker, Clark, Maier, & Viger, 2008; Marks, 2000; Weiss et al., 2010), including lower levels of problem behaviors (Appleton et al., 2008; Weiss et al., 2010). Increased student engagement has also been shown to help students attain their educational goals, regardless of their racial or ethnic background (Kuh, 2009; Marks, 2000) or socio-economic status (Sharkey, You, & Schnoebelen, 2008). Those students who are engaged in school are much more likely to have better outcomes in adulthood (Sharkey et al., 2008), including pursuing and doing well in higher education (Kuh, 2009; Marks, 2000). This success is due in part to the persistence (Caraway & Tucker, 2003; Kuh, 2009; Linnenbrink & Pintrich, 2003; McClenney, 2007), self-confidence (Caraway & Tucker, 2003; Linnenbrink & Pintrich, 2003; Sharkey et al., 2008; Uekawa, Borman, & Lee, 2007), and goal-setting (Caraway & Tucker, 2003; Sharkey et al., 2008) that are associated with student engagement. These traits may be especially beneficial for students that might otherwise be at high risk for not succeeding (McClenney, 2007), and apply to both male and female students, irrespective of ethnic background, that are highly engaged at the elementary, middle, secondary, and college levels (Linnenbrink & Pintrich, 2003).

What Is the Definition of Student Engagement?

How can the construct of “student engagement” be defined? Appleton et al. (2008) stated, “The theoretical and research literatures on engagement generally reflect little consensus about definitions and contain substantial variations in how engagement is operationalized and measured” (p. 370). A literature review revealed common components, usually ranging from two to four in number. Weiss et al. (2010) defined student engagement as having two dimensions: behavioral, which addresses student preparedness, attendance, and time spent on task; and psychological, which is described as student enthusiasm, interest, and identity. Archambault, Janosz, Fallu, & Pagani (2009a) define behavioral engagement as student conduct that “is beneficial to psychosocial adjustment and achievement at school” (p. 653). The concept of internal motivation, which reflects students’ personal and intrinsic interest, the value students place on content or a task, and their value beliefs about the importance of content or a task (Linnenbrink & Pintrich, 2003), represents a more covert dimension of engagement. Motivation could be driven by self-efficacy (Linnenbrink & Pintrich, 2003), but is considered separate from engagement by others (Appleton et al., 2008), who argue that just because a student is motivated does not mean that they will be actively engaged in a task.

Student engagement has also been defined as a three-component model, which encompasses behavioral, cognitive, and affective or emotional aspects (Appleton et al., 2008; Archambault et al., 2009a; Archambault, Janosz, Morizot, & Pagani, 2009b; Caraway & Tucker, 2003; Tucker et al., 2002). Since behavior can be observed, the behavioral dimension encompasses attendance, student actions, and participation (Archambault et al., 2009b; Appleton et al., 2008). The cognitive dimension, similar to motivation, concerns student perceptions of competency, willingness to engage in class activities, and self-regulation strategies (Archambault et al., 2009b). The affective or emotional dimension refers to students’ feelings, interests, attitudes, and perceptions toward school (Archambault et al., 2009a; Archambault et al., 2009b).

A four-subtype model of engagement, comprised of academic, behavioral, cognitive, and psychological components, has also been proposed (Anderson, Christenson, Sinclair, & Lehr, 2004; Appleton et al., 2008; Reschly, Huebner, Appleton, & Antaramian, 2008). Academic is defined by grades, credits, and assignment completion, while psychological describes feelings and cognitive describes thinking (Appleton et al., 2008; Sharkey et al., 2008). Educators should not rely solely on overt or low-inference indicators, such as behaviors; cognitive engagement is not always apparent (Linnenbrink & Pintrich, 2003), and cognitive and psychological aspects of engagement can provide students with the emotional support needed to succeed (Reschly et al., 2008).

Other factors commonly associated with student engagement were identified in the research. A sense of belonging in the school environment, cultivated through attendance and participation, leading to acceptance and respect, was noted as key to student engagement (Anderson et al., 2004; Archambault et al., 2009a; Baker et al., 2008; Juvonen, 2007). Student effort and investment in learning was also viewed as very important to engagement, as it led to increased self-efficacy (Caraway & Tucker, 2003; Kuh, 2009; LaNasa, Cabrera, & Trangsrud, 2009; Marks, 2000; Spanjers, Burns, & Wagner, 2008). Goal orientation was identified as a common variable affecting engagement in the literature, as was student fear of failure (Caraway & Tucker, 2003).

Regardless of what definition is chosen for the construct of student engagement, it is important to keep in mind that “every school irrespective of school level, geographic locale, or demographic characteristics of students has students who are disengaged and engaged” (Appleton et al., 2008, p. 377). For example, the 2006 High School Survey of Student Engagement, or HSSSE, surveyed 81,499 students in grades 9-12 from 110 schools in 26 states; 72% of the student respondents indicated that they were engaged in school, meaning that over a quarter of the students surveyed were not engaged (Appleton et al., 2008). Comment by Jing Lei: In this section you provided a thorough review of definitions of student engagement. Nice research!

What Are the Effects of Disengagement?

Weiss et al. (2010) defined disengagement as being observable through students cutting class, tardiness, violence, and vandalism. Students exhibiting behavior problems typically have lower rates of academic engagement (Baker et al., 2008). Students that are disconnected from school are reported to have other variables that have influenced their disengagement, such as a fear of failure (Caraway & Tucker, 2003), having low intellectual skills, and a past history of underachievement in special education programs, as well as being male (Archambault et al., 2009b). Lack of engagement is viewed as a gradual process that can result in disciplinary problems, absenteeism, and eventual school dropout (Baker et al., 2008; Marks, 2000).

Through better understanding of the construct of student engagement, it could be possible to intervene in students’ lives before their disconnection leads to dropping out of school (Anderson et al., 2004; Appleton et al., 2008). Research has shown that students with higher levels of academic engagement are less likely to drop out of school before degree completion (Weiss et al., 2010). However, disengagement is a gradual process, possibly beginning as early as elementary school (Anderson et al., 2004; Archambault et al., 2009b). Engagement is viewed as the primary means of understanding dropout (Appleton et al., 2008), and variables such as preparation for class and school, attendance, and problem behaviors are predictors of school dropout and completion within groups of students that are at high risk for dropping out of school, such as low-income African-Americans and students with disabilities (Reschly et al., 2008). High school dropouts are more likely to experience problems such as less lifetime earnings, more risky health behaviors and attitudes, poorer mental health, teenage pregnancy, gang involvement, criminal activity, and substance use (Archambault et al., 2009b; Caraway & Tucker, 2003). Even excluding dropouts and repeated absenteeism, disengagement is widespread: Marks (2000) stated that chronic disengagement reportedly affects 40-60% of secondary school students.

How Is Student Engagement Measured? Comment by Jing Lei: This is a key question!

Various instruments have been used to assess student engagement. In order to capture the cognitive and affective/emotional aspects of engagement, many of these instruments ask the students to self-report through questionnaires. The most prevalent measurement of student engagement, the National Survey of Student Engagement, or NSSE, is used at the college level (Kuh, 2009; LaNasa et al., 2009; National Survey of Student Engagement). The NSSE is based on seven principles of good practices in undergraduate education (LaNasa et al., 2009), and relies on five benchmarks as indicators of student engagement: 1) Level of Academic Challenge; 2) Active and Collaborative Learning; 3) Student-Faculty Interaction; 4) Enriching Educational Experiences; and 5) Supportive Campus Environment (Kuh, 2009; LaNasa et al., 2009). LaNasa et al. (2009) explored the construct validity of this instrument, and determined an eight factor model was better suited to the measurement: 1) Learning Strategies; 2) Academic Interaction; 3) Institutional Emphasis; 4) Co-curricular Activity; 5) Diverse Interactions; 6) Effort; 7) Overall Relationships; and 8) Workload. This model was purported to fit the data better, especially when aggregated across multiple institutions (LaNasa et al., 2009).

Several instruments similar to the NSSE are also in use. The Community College Survey of Student Engagement, or CCSSE, was established in 2001 (McClenney, 2007), and has been validated through three independent studies as a demonstration of students’ academic achievement and persistence (Kuh, 2009). The CCSSE results are reported according to five benchmarks, quite similar to the NSSE: 1) Active and Collaborative Learning; 2) Student Effort; 3) Academic Challenge; 4) Student-Faculty Interaction; and 5) Support for Learners (McClenney, 2007). Other instruments that are based on the NSSE include the College Student Experiences Questionnaire (CSEQ, which measures students’ “quality of effort”), the Classroom Survey of Student Engagement (CLASSE), the Law School Survey of Student Engagement (LSSSE), the Community College Faculty Survey of Student Engagement (CCFSSE), the Faculty Survey of Student Engagement (FSSE), the Beginning College Survey of Student Engagement (BCSSE, for four-year schools), and the Survey of Entering Student Engagement for Community Colleges (SESECC) (Beginning College Survey of Student Engagement; Classroom Survey of Student Engagement; Kuh, 2009).

Appleton et al. (2008) noted that there are few instruments that measure student engagement and equate with expected outcomes; two instruments that are exceptions to this are the High School Survey for Student Engagement (HSSSE) and the Student Engagement Instrument (SEI) developed by Appleton, Christenson, Kim, and Reschly (2006). The HSSSE addresses three dimensions: cognitive/intellectual/academic (i.e., engagement of the mind); social/behavioral/participatory (i.e., engagement in school life); and emotional (i.e., engagement of the heart) (Appleton et al., 2008). Some of the categories included in the HSSSE are: Foundations of Engagement (why students go to school; boredom and engagement; dropping out and disengagement); Student Actions for Learning (time-on-task and task importance; effort); Rigor and Relevance (challenge of classes; focus of work; contribution to growth); Relationships, Support, and Connection (belief of teachers; support from adults and peers; safety and fairness; connection to school community); and Instructional Methods and Pedagogical Possibilities (High School Survey of Student Engagement; Yazzie-Mintz, 2009). The SEI delves more into students’ personal competency beliefs, goal orientation, and sense of belonging (Appleton et al. 2008). The dimensions addressed by the SEI include: Family Support for Learning; Peer Support for Learning; Teacher-Student Relationships; Control and Relevance of Schoolwork; and Future Aspirations and Goals (Appleton, Christenson, Kim, & Reschly, 2006; Appleton et al., 2008; Lewis, Huebner, Reschly, & Valois, 2009; Reschly et al., 2008).

Other self-report instruments have been used to determine student engagement. Tucker et al. (2002) had students in grades 1-12 rate themselves on five subscales of engagement: emotional engagement; centrality of school; effort; attention; and what they termed “beyond the call,” which determined how much students tried to learn about their subjects. Archambault et al. (2009a) used 18 self-report items to measure aspects of behavioral, affective, and cognitive engagement, while Sharkey et al. (2008) used a five-item scale constructed from items originally in the National Longitudinal Study of Adolescent Health to measure only psychological engagement. Marks (2000) defined student engagement in instructional activity as having four measures: student effort; attentiveness; lack of boredom in class; and completing class assignments. Finally, Weiss et al. (2010) identified seven variables that they believed functioned together as a composite measure of school engagement: 1) teacher experience (years of teaching experience); 2) delinquent student behavior (late for school, cut/skipped classes, in trouble, suspended, put on probation); 3) academic friend (the importance of scholastic grades to a student’s three closest friends); 4) educational motivation (a student’s perception of school importance); 5) teachers’ belief about ability (whether the teachers believe a student can learn); 6) school preparedness (extent to which a student arrives at school prepared to learn); and 7) parental involvement (parent-teacher organizations, volunteering in school; belonging to other parent organizations).

Direct observation methods were used in other studies to note student behaviors as they related to student engagement. Uekawa et al. (2007) observed and noted the amount of time in each class and how it was distributed among down time, group work, individual work, testing, and lecturing or reference. They compared their findings to more cognitive measures, which included the instructional topic, the relevance of the instructional topic, the students’ affective state, and classroom conversation (Uekawa et al., 2007). Baker et al. (2008) also conducted direct observation of children’s classroom behavior, and categorized student behaviors as either on-task (paying attention to instruction or actively engaging with the learning task) or off-task (not paying attention, disrupting the class, or not being actively engaged with the academic task). These behaviors were also interpreted according to the classroom context at the time of observation, which was coded into one of five primary categories: classroom disruption; classroom transition; individual seatwork; small-group instruction; and whole-group instruction (Baker et al., 2008).

What Are Factors that Affect Student Engagement?

Student engagement is greatly influenced by the environmental contexts in which the student operates (Appleton et al., 2008; Baker et al., 2008). Caraway and Tucker (2003) noted:

Many contextual and self-variables can either enhance and/or impede school engagement. Contextual variables refer to external influences in an adolescent’s life such as family support, peer relationships, school environment, or neighborhood characteristics. Self or internal variables are characteristics of an individual such as level of self-efficacy or autonomy. Contextual variables and self-variables frequently have a complex relationship with one another and are often closely related. Therefore, a theoretical perspective that integrates the important contributions of both types of variables can provide an organizing framework for better understanding the role of these variables in children’s school engagement. (p. 418)

The role of the school environment as a context is very important to student engagement. Student perceptions of the learning tasks, especially beliefs about lesson importance, utility, and relevance (Burks, Heidenberg, Leoni, & Ratliff, 2009; Linnenbrink & Pintrich, 2003; Uekawa et al., 2007), can greatly affect engagement. Instructional context can also affect engagement through school resources, discipline practices, and sense of community (Appleton et al., 2008; Baker et al., 2008; Weiss et al., 2010). Teacher involvement and support is reported to be positively related to student engagement (Juvonen, 2007; Tucker et al., 2002), and a teacher’s course execution, classroom environment, and personality may also affect a student’s achievement (Burks et al., 2009).

Social support as a whole for students, whether from teachers, parents, or peers, has been found to influence student engagement. Students with favorable social interactions are less likely to exhibit disengaged behaviors (Anderson et al., 2004; Juvonen, 2007; Marks, 2000). Supportive relationships from friendships and connections with teachers may also help psychological development (Sharkey et al., 2008) and sense of belonging, therefore increasing engagement (Juvonen, 2007; Marks, 2000). Parental support and behaviors are also strongly related to student engagement, as the home environment is where children are exposed to management skills, discipline techniques, and monitoring strategies (Sharkey et al., 2008).

Another factor that affects student engagement is their concept of self-efficacy. A student’s fear of failure about themselves and their ability to perform well in a content area would affect their classroom performance (Burks et al., 2009). Motivational and affective barriers may otherwise exist within a student, and they must be understood in order to help them succeed (Appleton et al., 2008; Burks et al., 2009; Linnenbrink & Pintrich, 2003; Sharkey et al., 2008; Tucker et al., 2002). Reschly et al. (2008) reported that students exhibiting frequent positive emotions during school were more likely to also have higher levels of engagement.

Students that exhibit early behavioral problems may not have the skills needed for successful engagement (Baker et al., 2008). However, students’ behaviors, especially attendance, preparation, work completion, and eagerness to learn, may be easier to impact than their social interactions at school (Anderson et al., 2004). Since indicators of the behavioral dimension of student engagement can be used to predict dropout rates (Archambault et al., 2009a), teachers should strategically create classroom contexts that support their students’ academic engagement (Baker et al., 2008).

Finally, there are factors associated with high-risk students that may affect their engagement. Sociodemographic statistics have been shown to be significantly related to student engagement (Weiss et al., 2010), and Marks (2000) found that social class contributed significantly to engagement at the elementary, middle, and secondary levels. Lower socio-economic status (SES) students may not understand or relate to the values and views that are presented in the classroom (Uekawa et al., 2007), and lower levels of family and school assets have been related to lower levels of student engagement (Reschly et al., 2008; Sharkey et al., 2008). However, Marks (2000) reported that “SES consistently predicts engagement for middle school students only. For high school students, the SES effect is significant only in the personal background model” (p. 171). Research is conflicting on the impact of student ethnicity on engagement; Appleton et al. (2008) identify ethnicity and home language as variables for students at high risk for school failure, although student participation and identification with school can increase engagement and achievement. Weiss et al. (2010) did not find significant difference between White and African-American students and their engagement, and Marks (2000) found no racial or ethnic effect on engagement at any grade level, yet noted that “minority students from low-income homes tend to be disengaged in the classroom” (p. 156).

How Can Student Engagement Be Increased?

Due to the multidimensional nature of the construct of student engagement, there can be many different ways to address engagement issues. Several studies support understanding the students and their related self and contextual variables in order to increase their engagement (Archambault et al., 2009a; Archambault et al., 2009b; Caraway & Tucker, 2003). Reschly et al. (2008) observed that engagement has been found to vary based on gender: girls tend to be significantly more academically engaged, yet the influence of relatedness on engagement has a greater impact for boys. Regardless of differences, the goal of increasing student engagement is “ultimately intended to increase students’ achievement” (Weiss et al., 2010, p. 165). How can this be done?

One intervention program is called Check & Connect; the check is based on indicators of engagement such as attendance, behavior referrals, and academic progress; the connect refers to personal connections that the program’s monitors make with students, school staff, and families (Anderson et al., 2004). This program addresses the behavioral, cognitive, and affective/emotional dimensions of student engagement (Archambault et al., 2009b), which it promotes through relationship building, persistence, and problem solving (Anderson et al., 2004).

Besides formal intervention programs, classroom strategies and the school environment as a whole can be modified to increase student engagement. Uekawa et al. (2007) suggested hands-on problem-solving activities and peer collaboration, as teachers provided social organizational variables for students that affect both their behaviors and attitudes. Baker et al. (2008) reported that effective instructional practices have been shown to increase student engagement and support positive behaviors, especially for those students with behavioral problems. Suggestions included shorter task duration, modifying task sequencing, providing frequent reinforcement, and matching task demands to student skill level (Baker et al., 2008). Other instructional practices were also recommended, such as giving clear instructions when introducing tasks, offering support to facilitate task completion, monitoring students’ levels of engagement, stopping off-task behavior quickly, providing highly structured classroom transitions, and preventing unnecessary distractions during instructional time (Baker et al., 2008). In addition, teachers can provide authentic learning experiences that are relevant to students to help increase engagement (Burks et al., 2009; Marks, 2000). Teachers can also foster students’ self-efficacy by helping them maintain academic confidence, and providing them with challenging tasks that they can ultimately complete (Linnenbrink & Pintrich, 2003).

Plan for Seeking Solutions

There are many factors that could be contributing to the lack of student engagement at PWHS. In order to be able to address the problem as a whole, these dimensions need to be understood. The literature about student engagement acknowledges at least behavioral and cognitive, and sometimes affective/emotional, components to student engagement, and also emphasizes the importance of context to student achievement. Harless (1970) categorized human performance problems into three categories: skill/knowledge; environmental; and motivation/incentive. All of these categories fit well with the research surrounding student engagement; perhaps through using this framework, a front-end analysis (FEA) can be planned that helps to ameliorate PWHS’s student engagement problem. Based on the literature, environmental context is likely playing the greatest role in affecting student engagement. However, student behaviors (exhibited by attendance and grades) may be indicators of skill/knowledge and motivation/incentive, and according to the research, are part of the component of student engagement that may be most amenable to modification.

Selection of Approaches, Procedures, and Instrumentation for Analysis

In order to determine the current state of students at PWHS and to determine what their desired state should be, various data collection instruments should be used. Since student engagement is a construct based on many different variables, it will be very important to understand the specific influences that affect engagement at PWHS before proceeding with any intervention strategies (Rossett, 1987).

Questionnaires

In order to learn more about students’ overall opinions about school, a survey questionnaire can be used. The HSSSE may be chosen to be used by the school, or another instrument may be developed. Regardless of what measurement is used, information about students’ self-efficacy, time-on-task, motivation at school, peer relationships, teacher relationships, school environment, and family background should be obtained. An overview of student perceptions and how they relate to the previous research on student engagement will be crucial in understanding what factors are most important at PWHS.

Questionnaires can also be used with teachers and administrators to learn more about the academic context. An instrument can be adapted from faculty questionnaires that already exist at the collegiate level, such as the CCFSSE and the FSSE, as well as a student survey for the high school level, such as the HSSSE. Questions should be asked about teacher experience, instructional strategies used in the classroom, teacher-student relationships, perceived student efficacy, and other factors in the school environment that may contribute to student engagement.

The responses from all questionnaires can be coded according to the specific dimensions that are addressed within the instrument. The reviewed results can serve as a basis for guiding other data collection methods, such as focus groups, interviews, and on-site observations.

Focus Groups

Focus groups can be used in conjunction with questionnaires to understand more about the cognitive and affective/emotional aspect involved in student engagement at PWHS. Students and teachers can be interviewed separately to talk about what they feel about the school environment, whether they think students are successful at the school, what factors they think affect student performance, and what interventions they believe have worked at the school to help student engagement.

The data from the focus groups can be coded for frequency. They will help illuminate results from the questionnaires, and can also guide the direction of individual interviews.

Interviews

Interviews can be conducted with individual students and teachers to learn more about the actual classroom practices that exist at PWHS. They can also help to provide more information to clarify the findings from the questionnaires and focus groups about student engagement at the school. Students will be asked about their performance at the school, their self-perceptions of ability, the social relationships that they have in school, and their overall interest in academic subjects. Teachers can be queried about the school environment, how they perceive their students to be performing, overall behaviors in their classrooms, and what kind of instructional strategies they employ and why.

Data from the interviews can be compared to questionnaire and focus group results to ensure that student engagement at PWHS is being accurately represented. Like the focus groups, the interview data will be qualitatively analyzed and coded for frequency in order to make these comparisons.

On-Site Observations

Because self-report data is not always the most reliable, the analyst and their team should conduct on-site observations at the school. These observations will primarily be done in the classroom, but can also be used to understand the school environment as a whole. An observation checklist of behaviors will need to be developed prior to these site visits. For the classroom, the observer can monitor students’ participation in class, time on task in both individual and group work, and response to teacher questions. Teachers can be monitored for the instructional strategies they use, whether their lessons are differentiated to meet students’ needs, and whether they support students through challenging learning tasks that students can ultimately achieve. For school-wide observations, interactions between teachers and administrators and students can be observed, as well as disciplinary issues that the school may encounter.

The data from the on-site observations can be triangulated with the qualitative and quantitative data that is collected to ensure that a depiction of student engagement at the school is accurate. Observations may also be compared to existing literature on observable measures of student engagement.

Records Analysis

Finally, records analysis can be used in order to learn more about students’ behavioral and academic engagement. Attendance and truancy records and discipline records and statistics can inform the behavioral aspect of student engagement. Students’ course grades, standardized test scores, and AYP data can provide more information about academic engagement. Extant data on pre- and post-intervention strategies can also be used to learn more about what was effective in increasing overall student performance.

As with all other proposed data collection methods, extant records analysis can help to describe the school environment and its possible impact on student engagement. The records analysis may be performed prior to the creation of other instrumentation, or may be used concurrently to help elucidate the current state of PWHS.

Possible Causes

Below are hypothesized causes of lack of student engagement at PWHS. The causes are sorted according to Harless’s (1970) categories of human performance problems.

Knowledge and Skill

· Lack of student abilities (students do not know the material)

· Students have limitations that affect performance (special education needs, disabilities, English is a second language)

· Lack of teacher abilities (teachers are using ineffective instructional strategies)

Environment

· Relatively high student-to-teacher ratio (30 or more students in math and English/Language Arts classes)

· Intervention programs are attempted for less than a few years before they are abandoned

· Ineffective intervention programs

· Lack of administrative organization

· Lack of parental support and involvement

· Lack of community support and involvement

· School environment as a whole not conducive to learning

· Lack of peer academic support for students

· Lack of teacher support for students

· Lack of social/psychological support for students

· Frequent disruptions due to discipline problems

· Focus on standardized testing to meet AYP, rather than on lessons

Motivation and Incentive

· Student frustration with standardized testing

· Student attendance problems

· Students’ fear of failure

· Lack of strong teacher-student relationships

· Lack of relevance of school to students’ lives

· Students feel they cannot do the schoolwork

· Students feel they are not challenged by the schoolwork

· Frequent disruptions due to discipline problems

Possible Solutions

Given the hypothesized causes above, what are potential solutions? Again, these are categorized according to Harless’s (1970) FEA plan model. Note that some suggestions appear multiple times.

Knowledge and Skill

· Tutoring programs for students

· More support for students with special needs

· Professional development to help teachers learn and implement effective instructional strategies

Environment

· Reduce class size, if possible, or allow for multiple teachers in a classroom with a large number of students to facilitate personal attention

· Commit to intervention programs once they have been implemented; thoroughly research programs and their potential usefulness to PWHS before they are put into action so that time, effort, and resources are not spent restructuring how instruction is delivered

· Increase parental and community involvement with school, possibly through community events and back-to-school night events that are more conducive to family schedules

· Publicize positive happenings at the school in the media, rather than only allowing negative information to be distributed

· Increase communication between administration, teachers, parents, and community to help organization

· Develop strict discipline policies, and hold all students and teachers accountable to them

· Provide more academic and psychological support for students in the form of tutoring and counseling

· Create academic clubs and groups that engage students in learning outside of school

· Address standardized testing as needed in the classroom without losing sight of curriculum objectives

· Professional development to help teachers learn how to support and encourage students, and how to challenge them academically with instructional tasks they can achieve

· Professional development to help teachers use consistent and effective instructional strategies

Motivation and Incentive

· Incorporate standardized testing preparation into required lessons, so the exams do not dominate class time

· Provide incentives for students who have strong attendance records

· Provide more academic and psychological support for students in the form of tutoring and counseling to help increase their self-efficacy

· Differentiate instruction so that it is well-suited to all students, not just those in the middle range of ability

· Use student input to help increase relevance of coursework to students’ lives

· Develop strict discipline policies, and hold all students and teachers accountable to them

What Is the Best Possible Solution?

There are certainly many different approaches that could be taken to address low student engagement at PWHS. Although plans for addressing motivation/incentive and knowledge/skills issues are important, the primary problem seems to be school environment. Making changes within the school environment can improve the overall instructional context, which will affect student performance (please refer to What Are Factors that Affect Student Engagement?, above). Creating an environment much more conducive to learning can help students develop stronger self-efficacy, as they are more likely to receive academic support. Struggling students are also more likely to receive the help that they need in a strong academic environment, and teachers will likely be more willing to take on changes in their pedagogies if it is to the benefit of the entire school.

Final Presentation of FEA Plan

The FEA plan, as described above, would be shared with the administration of the school, as well as other key stakeholders, such as department chairs. Although the plan will eventually need to be presented to and approved by the district administration (superintendent and school board), the plan must be viewed as feasible and worthwhile by those at PWHS, as it is their school that ultimately will be impacted. The school-level stakeholders are also much more familiar with their own school environment, and can provide constructive feedback to the analyst. This will help guide them in an effective implementation of an FEA plan that yields results that are ultimately beneficial to the school and its students through an increase in student engagement.

References

Anderson, A. R., Christenson, S. L., Sinclair, M. F., & Lehr, C. A. (2004). Check & Connect: The importance of relationships for promoting engagement with school. Journal of School Psychology, 42(2), 95-113.

Appleton, J. J., Christenson, S. L., & Furlong, M. J. (2008). Student engagement with school: Critical conceptual and methodological issues of the construct. Psychology in the Schools, 45(5), 369-386.

Appleton, J. J., Christenson, S. L., Kim, D., & Reschly, A. L. (2006). Measuring cognitive and psychological engagement: Validation of the student engagement instrument. Journal of School Psychology, 44(5), 427-445.

Archambault, I., Janosz, M., Fallu, J.-S., & Pagani, L. S. (2009a) Student engagement and its relationship with early high school dropout. Journal of Adolescence, 32(3), 651-670.

Archambault, I., Janosz, M., Morizot, J., & Pagani, L. (2009b). Adolescent behavioral, affective, and cognitive engagement in school: Relationship to dropout. Journal of School Health, 79(9), 408-415.

Baker, J. A., Clark, T. P., Maier, K. S., & Viger, S. (2008). The differential influence of instructional context on the academic engagement of students with behavior problems. Teaching and Teacher Education 24(7), 1876-1883.

Beginning College Survey of Student Engagement (BCSSE). Retrieved from

http://bcsse.iub.edu/survey_instruments_2009.cfm

Burks, R., Heidenberg, A., Leoni, D., & Ratliff, T. (2009). Supporting the motivators: A faculty development issue. PRIMUS, 19(2), 127-145.

Caraway, K., & Tucker, C. M. (2003). Self-efficacy, goal orientation, and fear of failure as predictors of school engagement in high school students. Psychology in the Schools, 40(4), 417-427.

Classroom Survey of Student Engagement (CLASSE) Student Survey. Retrieved from

http://www.assessment.ua.edu/CLASSE/Documents/CLASSE_Student.pdf

Harless, J. (1970).Steps in front-end analysis. In IDE 712 Module II (pp. 30-32). Syracuse, NY: Syracuse University.

High School Survey of Student Engagement (HSSSE). Retrieved from http://www.indiana.edu/~ceep/hssse/about.html

Juvonen, J. (2007). Reforming middle schools: Focus on continuity, social connectedness, and engagement. Educational Psychologist, 42(4), 197-208.

Kuh, G. D. (2009). What student affairs professionals need to know about student engagement. Journal of College Student Development, 50(6), 683-706.

LaNasa, S. M., Cabrera, A. F., & Trangsrud, H. (2009). The construct validity of student engagement: A confirmatory factor analysis approach. Research in Higher Education, 50(4), 315-332.

Lewis, A. D., Huebner, E. S., Reschly, A. L., & Valois, R. F. (2009). Journal of Psychoeducational Assessment, 27(5), 397-408.

Linnenbrink, E. A., & Pintrich, P. R. (2003). The role of self-efficacy beliefs in student engagement and learning in the classroom. Reading and Writing Quarterly: Overcoming Learning Difficulties, 19(2), 119-137.

Marks, H. M. (2000). Student engagement in instructional activity: Patterns in the elementary, middle, and high school years. American Education Research Journal, 37(1), 153-184.

McClenney, K. M. (2007). The Community College Survey of Student Engagement. Community College Review, 35(2), 137-146.

National Center for Education Statistics. Penn Wood Senior High School [Data file]. Retrieved from http://nces.ed.gov/ccd/schoolsearch/school_detail.asp?Search=1&InstName=penn+wood&State=42&Zip=19050&SchoolType=1&SchoolType=2&SchoolType=3&SchoolType=4&SpecificSchlTypes=all&IncGrade=-1&LoGrade=-1&HiGrade=-1&ID=422639001892

National Survey of Student Engagement (NSSE). Retrieved from http://nsse.iub.edu/

Pennsylvania Department of Education. 2003-2009 School AYP Status [Data file]. Retrieved from http://www.portal.state.pa.us/portal/server.pt/gateway/PTARGS_0_123031_595904_0_0_18/AYP_School_2009.pdf

Pennsylvania Department of Education—Academic Achievement Report. AYP Status Levels. Retrieved from http://paayp.emetric.net/Content/datafiles/PDE%20AYP%20Flowchart.pdf

Pennsylvania Department of Education—Academic Achievement Report. Retrieved from http://paayp.emetric.net/

Pennsylvania Department of Education. 2008-2009 PSSA and AYP Results. Retrieved from http://www.education.state.pa.us/portal/server.pt/community/school_assessments/7442

Pennsylvania Department of Education. The 2009 PSSA Mathematics and Reading School Level Proficiency Results (by Grade and School Building) [Data file]. Retrieved from http://www.portal.state.pa.us/portal/server.pt/gateway/PTARGS_0_123031_595945_0_0_18/PSSA_Results_Math_and_Reading_School_2009.pdf

Reschly, A. L., Huebner, E. S., Appleton, J. J., & Antaramian, S. (2008). Engagement as flourishing: The contribution of positive emotions and coping to adolescents’ engagement at school and with learning. Psychology in the Schools, 45(5), 419-431.

Rossett, A. (1987). Training needs assessment. Englewood Cliffs, NJ: Educational Technology Publications.

Sharkey, J. D., You, S., & Schnoebelen, K. (2008). Relations among school assets, individual resilience, and student engagement for youth grouped by level of family functioning. Psychology in the Schools, 45(5), 402-418.

Spanjers, D. M., Burns, M. K., & Wagner, A. R. (2008). Systematic direct observation of time on task as a measure of student engagement. Assessment for Effective Intervention, 33(2), 120-126.

Tucker, C. M., Zayco, R. A., Herman, K. C., Reinke, W. M., Trujillo, M., Carraway, K., Wallack, C., & Ivery, P. D. (2002). Teacher and child variables as predictors of academic engagement among low-income African American children. Psychology in the Schools, 39(4), 477-488.

Uekawa, K., Borman, K., & Lee, R. (2007). Student engagement in U.S. urban high school mathematics and science classrooms: Findings on social organization, race, and ethnicity. The Urban Review, 39(1), 1-43.

Weiss, C. C., Carolan, B. V., & Baker-Smith, E. C. (2010). Big school, small school: (Re)testing assumptions about high school size, school engagement and mathematics achievement. Journal of Youth and Adolescence, 39(2), 163-176.

Yazzie-Mintz, E. (2009). Engaging the voices of students: A report on the 2007 & 2008 High School Survey of Student Engagement. Retrieved from http://ceep.indiana.edu/hssse/images/HSSSE_2009_Report.pdf