Big Brothers and Big Sisters of America
HermineG!
retrieve2.pdf
Mentoring in Schools: An Impact Study of Big Brothers
Big Sisters School-Based Mentoring
Carla Herrera, Jean Baldwin Grossman, Tina J. Kauh, and Jennifer McMaken Public ⁄ Private Ventures
This random assignment impact study of Big Brothers Big Sisters School-Based Mentoring involved 1,139 9- to 16-year-old students in 10 cities nationwide. Youth were randomly assigned to either a treatment group (receiving mentoring) or a control group (receiving no mentoring) and were followed for 1.5 school years. At the end of the first school year, relative to the control group, mentored youth performed better academically, had more positive perceptions of their own academic abilities, and were more likely to report having a ‘‘spe- cial adult’’ in their lives. However, they did not show improvements in classroom effort, global self-worth, relationships with parents, teachers or peers, or rates of problem behavior. Academic improvements were also not sustained into the second school year.
As parents strive to raise children who succeed aca- demically, develop supportive bonds with others and avoid problem behaviors, they meet many challenges. These challenges are particularly salient in homes where positive adult role models are not consistently available to help children make healthy decisions. Mentoring programs aim to fill such gaps in children’s lives, by matching volunteers with youth who could benefit from extra support and guidance. These programs have grown rapidly since the mid-1990s, bolstered by a series of studies showing that mentoring improves the lives of youth (Aseltine, Dupre, & Lamlein, 2000; Grossman & Tierney, 1998; LoSciuto, Rajala, Townsend, & Taylor, 1996).
While all forms of mentoring are increasing in prevalence, school-based mentoring (SBM) is grow- ing particularly rapidly. In 2005, approximately 870,000 adults were mentoring children in schools (MENTOR, 2006). In SBM, mentors typically build relationships with youth by meeting one-on-one with them at their school for about an hour a week during or after the school day, engaging in a wide range of academic and nonacademic activities.
SBM has grown, in part, due to increasing con- cerns about student performance, and schools’ efforts to implement interventions that might address their students’ challenges and foster aca- demic success. Students between the ages of 9 and 14, in particular, experience major developmental and school-related changes that make them vulner- able to academic, social, and behavioral problems. Fourth grade, for example, is a time of profound pedagogical change in most school systems. In Grades 1 through 3, students learn to read; in fourth grade, students begin to read to learn (Chall, 1983). If children leave the third grade as poor readers and their reading skills do not improve over the summer, they quite likely will experience ongoing learning problems (Spreen, 1978, 1988). Fourth graders experiencing these problems need to be encouraged to stay engaged in school and address their reading challenges. During late ele- mentary school, children also begin developing a sense of their own competence and start honing
This study was conducted as part of a larger Public ⁄ Private Ventures evaluation funded by generous grants from The Atlan- tic Philanthropies (to Big Brothers Big Sisters of America), Philip Morris USA, and The William T. Grant Foundation. The Edna McConnell Clark Foundation also supported the study by foster- ing communication among key stakeholders during all phases of the project. We are very grateful to staff from the 10 Big Brothers Big Sisters agencies involved in the study, as well as the men- tors, youth and teachers who completed our surveys. We thank Eric Foster, Mike Barr, and Dareth Noel at the Institute for Sur- vey Research at Temple University for their data collection efforts, and the school staff who assisted them. Keoki Hansen and Joe Radelet at Big Brothers Big Sisters of America were incredibly supportive partners throughout the study’s imple- mentation, as was our advisory group: Amanda Bayer, David DuBois, Michael Karcher, Steven Liu, and Jean Rhodes. We also thank the anonymous reviewers for helpful feedback that shaped the final draft of this report.
Correspondence concerning this article should be addressed to Carla Herrera, Senior Policy Researcher, Public ⁄ Private Ventures, 2000 Market Street, Suite 550, Philadelphia, PA 19103. Electronic mail may be sent to [email protected].
Child Development, January ⁄ February 2011, Volume 82, Number 1, Pages 346–361
� 2011 The Authors
Child Development � 2011 Society for Research in Child Development, Inc.
All rights reserved. 0009-3920/2011/8201-0023
DOI: 10.1111/j.1467-8624.2010.01559.x
their social comparison skills, enabling them to compare their accomplishments and failures to those of their peers. Youth who are experiencing learning problems often feel inadequate at school relative to their peers and frequently disengage from academic work, resulting in a cycle of school failure (Finn, 1989). These students may also start acting out to get attention from their teachers and peers (Jimerson, Egeland, & Sroufe, 2000; Laird, Jor- dan, Dodge, Pettit, & Bates, 2001). In contrast to children who feel competent, those who suffer from learning problems in elementary school are more likely to be held back a grade or to drop out of high school (Alexander, Entwisle, & Horsey, 1997; Cairns, Cairns, & Neckerman, 1989). When youth enter junior high school, they also experience major changes in school structure and adult and peer rela- tionships. These shifts are associated with further deteriorations in self-confidence and academic engagement in many students (Eccles & Midgley, 1989; Seidman, Allen, Aber, Mitchell, & Feinman, 1994).
Many youth are able to overcome such chal- lenges because they have the adult support and skills needed to negotiate them. However, not all children have these resources. Mentoring programs aim to provide a key protective factor—a caring adult—to youth to help ensure that they can negoti- ate some of these developmental challenges. Youth involved in high-quality, secure relationships are more independent, more persistent and more socially competent (Bergin & Bergin, 2009)—charac- teristics that are important for healthy socio- emotional and academic development. Because children spend almost a third of their waking hours in school (Timmer, Eccles, & O’Brien, 1985) acquir- ing academic skills, values and behaviors, and forming relationships with adults and peers that can have a profound effect on their development, schools would appear to be an ideal context in which to provide youth with this type of relationship.
Several studies have been conducted to outline the effects of ‘‘traditional’’ community-based men- toring (CBM) that is implemented outside of the school context. For example, a meta-analysis by DuBois, Holloway, Valentine, and Cooper (2002) of 55 studies of mentoring programs (a large propor- tion of which were CBM programs) found modest but significant positive effects of mentoring on risky behavior, social competence, and academic and career outcomes. However, surprisingly little research has been conducted to determine whether this major new variant of SBM is similarly able to contribute to youth’s positive development. On the
one hand, SBM has several characteristics that could make it less effective than CBM. For example, SBM meetings are more brief than those in CBM (approximately 1 hr a week compared to 3 or 4 hr a week in CBM), the matches are shorter in duration (Herrera, Sipe, McClanahan, with Arbreton, & Pep- per, 2000) and the relationships developed in these programs seem to be less strong than those in CBM (Herrera, 2004; Herrera et al., 2000). Given that longer and stronger relationships yield bigger impacts (Grossman & Johnson, 1999; Grossman & Rhodes, 2002; Slicker & Palmer, 1993), one might expect smaller or fewer impacts for SBM.
On the other hand, the school-based nature of SBM interactions may prime the mentoring rela- tionship to operate through different pathways that may help youth in a range of academic, social, and behavioral areas—particularly those evident in the school environment. For example, SBM may be a strong intervention for improving youth’s relation- ships with teachers. The focused attention and interaction that mentors provide may help improve the student’s social skills in interactions with oth- ers, including teachers. School-based mentors also may focus a teacher’s attention on the youth and help realign the youngsters’ attitudes toward teach- ers. Rhodes, Grossman, and Resch (2000) found that mentoring improved youth’s perception of other adult relationships—in particular, the parent– child relationship. Although SBM may also affect the parent–child relationship, school-based mentors have more direct connection and communication with children’s teachers than with their parents (Herrera et al., 2000). Thus, we postulate that SBM may have even stronger effects on youth’s relation- ships with their teachers than those with their parents.
The hypothesized impact on both parental and teacher relationships should then catalyze other improvements in youth’s outcomes. For example, better parental relationships are associated with higher general, academic, and social self-concepts and lower levels of problem behavior (Lau & Leung, 1992). Strong teacher–student relationships can similarly influence: children’s motivation to achieve in school (Patrick, Anderman, & Ryan, 2002), their interest in, enjoyment from and valuing of schoolwork (Fraser & Fisher, 1982; Jacobs, Lanza, Osgood, Eccles, & Wigfield, 2002; Midgley, Feldlau- fer, & Eccles, 1989), their expectations for school achievement (Goodenow, 1993), and their school performance (Hamre & Pianta, 2001).
By helping to improve youth’s social skills and sense of self, SBM may work similarly in improving
School-Based Mentoring 347
youth’s peer relationships. Research further sug- gests that peers (particularly in elementary school) may see attention from a school-based mentor in a very positive light, boosting the status of mentored youth (Hughes, Cavell, Meehan, Zhang, & Collie, 2005). Improving peer relationships early in devel- opment can then be critical in helping older adoles- cents stay out of trouble and stay in school (Parker, Jeffrey, & Asher, 1987).
By being located at school, the mentoring rela- tionship can also provide the child with a more positive experience and outlook on school. Studies show that participation in school-based activities increases students’ sense of school belonging and liking (Eccles & Barber, 1999; Grossman et al., 2002). This experience may in turn lead to improved attendance and academic performance.
Prior research, in fact, suggests that SBM partici- pation is associated with positive outcomes and that these associations differ from those found in CBM, focusing mostly on school and social domains and less on out-of-school behavior. For example, studies have found positive associations between SBM and academic performance (Diversi & Mecham, 2005; Hansen, 2001, 2002), self-perceptions of academic abilities (Bernstein, Dun Rappaport, Olsho, Hunt, & Levin, 2009) and attitudes toward school (Karcher, Davis, & Powell, 2002; King, Vido- urek, Davis, & McClellan, 2002; Portwood & Ayers, 2005). Studies have also found associations between SBM and improvements in peer relationships (Cavell & Hughes, 2000; Herrera, 2004; Karcher, 2008; King et al., 2002), attitudes toward parents (Karcher, 2005; Karcher et al., 2002), and self-esteem or self-confidence (Karcher, 2008; Matzenbacher, 1999). Few studies have examined out-of-school risky behaviors but some find reductions in school misbehavior (Cavell & Hughes, 2000; Matzenbach- er, 1999) and truancy (Bernstein et al., 2009). While these are all hopeful findings, most are based on non- or quasi-experimental evidence. To date, few large-scale experimental evaluations of SBM exist. Karcher’s (2008) and Bernstein et al.’s (2009) recent studies are notable exceptions.
SBM holds great potential for the mentoring field to yield benefits for youth, particularly in school- related areas during a developmental period when youth may be in need of relational and academic supports. If SBM can improve youth’s experiences and performance in school, its widespread use could foster the academic success of millions of children. However, if SBM is not effective, it will be important to outline its limitations so that school administrators, policymakers and funders can
redirect their resources into other proven strategies that may be more likely to enrich youth’s develop- ment.
To help the field make crucial decisions about where to invest its limited resources at this time of unprecedented growth, this study used a random assignment impact design to rigorously test the impacts of Big Brothers Big Sisters (BBBS) SBM both on outcomes closely linked with the school context and on more broad, out-of-school outcomes that previous studies have shown are related to participation in CBM programs. Namely, we examine impacts on school-related attitudes and performance, problem behavior in and outside of school, and social and personal well-being, includ- ing relationships with peers and adults and self- esteem. We anticipate that effects in out-of-school areas will not be as strong as those in school-related areas.
In addition, we explore whether program effects differ for youth of different ages, genders, and eth- nicities. We also examine whether youth reports of having a special relationship with an adult at base- line are associated with program benefits. On the one hand, we might expect that youth would receive a bigger boost from the intervention if the mentor is the only significant nonfamilial adult in the child’s life. On the other hand, research sug- gests that having already experienced a close rela- tionship with an adult prior to program participation may help prepare the child for creat- ing strong relationships with others (Rhodes, Con- treras, & Mangelsdorf, 1994). We explore these competing hypotheses by testing whether and in what ways program effects differ for youth with and without a special adult prior to program participation.
Method
Participants
Participants in the study include 1,139 youth who were in fourth through ninth grades at the start of the study in September 2004. Youth attended 71 schools that had SBM programs run by 1 of 10 geographically diverse BBBS agencies. On average, there were 16 participating youth per school, although this ranged from 1 to 101 partici- pating youth, and all but 5 schools had 35 or fewer participating youth. The agencies selected for this study had to have SBM programs that fulfilled sev- eral criteria. All programs: (a) had been operating for at least 4 years, (b) served at least 150 youth
348 Herrera, Grossman, Kauh, and McMaken
(both boys and girls), (c) recruited from at least two different types of volunteer populations (e.g., high school students, nearby employees), and (d) had strong relationships with school partners. The cho- sen 10 agencies represent a range of sizes, however on average, they are larger than the average BBBS agency, and they tended to have more experience with SBM and more supportive schools.
Our sample is 54% female and ranged in age from 8 to 18 years (M = 11.23), although over 99% of youth were between 9 and 16 years old. A total of 63% were minorities, with Latinos (23%), African Americans (18%), and multiracial youth (13%) comprising the largest minority groups. Sixty-nine percent received free or reduced-priced lunch dur- ing the 1st year of the study. Additionally, 39% lived in single-parent households. Sixty-one percent of participants were in fourth or fifth grade, about a third were in middle school (sixth through eighth grades), and 6% were ninth graders from three high schools served by one agency that targeted high school freshman at risk of dropping out. Teachers reported that about half (51%) of youth were either performing below grade level or needed improve- ment in their overall academic performance. Teach- ers also reported that only 12% had been involved in serious school infractions (i.e., fighting, suspen- sions, being sent to the principal’s office) in the 4 weeks prior to the baseline survey.
Of the 554 mentors who completed baseline sur- veys at the beginning of their program involve- ment, 72% were female and 77% were White. Approximately 44% of minority youth were matched with White mentors and 19% of youth were in cross-gender matches. Nearly half (48%) of mentors were high school students and 18% were college students at the time of the baseline survey. Mentors of different ages reported receiving fairly similar amounts and quality of training and sup- port over the course of their match (see Herrera, Kauh, Cooney, Grossman, & McMaken, 2008). Twenty-five percent of mentors had previous expe- rience mentoring in formal mentoring programs, and an additional 35% had prior informal mentor- ing experience.
Procedure and Intervention
Participating youth were referred to the program by school staff. Children who assented to partici- pate in the study (and had parents who gave their formal consent) were surveyed along with their teachers (all 1,139 youth and teachers of 1,009 youth [89%] completed baseline surveys). Baseline youth
surveys were administered at the child’s school by on-site researchers in small groups of 3–10 youth. Teacher surveys were self-administered. For youth in middle and high school who had multiple teach- ers, the child’s science, social studies, or homeroom teacher (or, if the child was learning English as a second language, the ESL teacher) was identified by the school liaison (i.e., a school staff member who worked with BBBS staff to recruit youth and oversee the program) and surveyed. These subjects were selected because science and social studies generally cover material that is relatively indepen- dent from the knowledge taught in those subjects in previous years. Thus, we hypothesized that aca- demic change might occur more quickly in these subjects than in math or language arts and these teachers might be able to observe change in school performance and behavior more quickly. Home- room teachers were also targeted because they see the students every day and might be aware of stu- dent’s performance in more than one subject area. If the youth did not have a teacher in any of these areas, another teacher was surveyed.
Once students completed their survey, they were randomly assigned to either the treatment group (which we refer to as ‘‘Littles’’) who were eligible to matched with a mentor (n = 565) or to the con- trol group (which we refer to as ‘‘their nonmen- tored peers’’) who were placed on agency waiting lists until the end of the study (n = 574). Random assignment was stratified by school so that the treatment and control groups within a given school were approximately the same size. However, because assignment to these groups was based on short sequences of pregenerated randomly ordered treatment designations rather than perfectly alter- nating patterns, the size of these groups is close to, but not exactly a 50–50 split.
Student and teacher surveys were collected in the spring of School Year 1, the ‘‘9-month assess- ment’’ (1,067 youth surveys, a 94% completion rate; 959 teacher surveys, an 84% completion rate), and in late fall of School Year 2, the ‘‘15-month assess- ment’’ (968 youth surveys, an 85% completion rate; 920 teacher surveys, an 81% completion rate). A total of 447 of 515 mentors in active matches also completed surveys at the 9-month assessment (an 87% completion rate). This survey included ques- tions about the match and the program; only infor- mation about the program is included here. Surveys for youth were administered by a survey firm at the schools or by phone for youth who had moved or were absent on the day of survey administration. Teacher surveys were, again,
School-Based Mentoring 349
self-administered. Mentor surveys were distributed by agency staff in eight agencies and by an outside survey firm in two agencies and were self-adminis- tered.
The agencies were responsible for recruiting and training the mentors. Mentors were recruited as they normally were in any other program year—typically from local businesses and high schools and in some cases, from colleges. Overall, 71% of mentors reported receiving training from the agency with an average duration of 1 hr. A majority of programs (80%) asked mentors to com- mit to meeting at least weekly with their Littles. Most mentors reported that their match meetings lasted either 45–60 min (40% of mentors) or over 1 hr (39%); only 21% reported having match meet- ings that lasted < 45 min.
Match meetings occurred in many different places on the school campus, typically in large spaces like the school cafeteria or library. About half (49%) of the programs in our sample operated during the school day, while 47% took place after school. The remaining programs (4%) held match meetings both during and after school. For 64% of matches, meetings involved interacting with other youth. These interactions often occurred in after- school programs in which matches typically met together in one space, but they also sometimes occurred in school-day programs, several of which met during lunch.
All of the programs participating in the study had some degree of structure (i.e., the activities from which matches could choose were, at least in part, outlined by the program), and in a few cases, the activities in which matches engaged were pre- determined by the school or BBBS. However, in most programs, matches chose how they spent their time together. For example, some programs offered suggestions for the meetings by providing a box of recreational activities from which mentors and their Littles could choose if they needed activity ideas. Reports from mentors point to a lack of a strong academic emphasis in the programs: Although most matches did engage in some academic activi- ties, only 27% spent ‘‘a lot’’ or ‘‘most’’ of their time engaging in tutoring or homework help. Instead, the matches engaged in a wide variety of other activities, including creative activities (e.g., draw- ing, arts and crafts), games and discussions about various issues and topics.
By the 9-month assessment, 93% of Littles had been matched with a mentor and had received an average of 4.9 months of mentoring, meeting an average of 3.1 times per month while their match
was active. By the start of the second school year, many of these matches had ended. Close to one third of the matches ended because the Little had transferred to a new school—either due to family mobility or ‘‘graduation’’ from elementary to mid- dle school or middle to high school. Although agency staff tried to find mentors for these youth, they were not always successful. In total, only 52% of Littles met with a mentor in the second school year—41% with their 1st-year mentor and an addi- tional 11% with a new mentor.
Measures
Outcome measures fell into three broad catego- ries: school-related performance and attitudes, problem behaviors, and social and personal well- being as described below. (Other outcome mea- sures were examined and the results of these analyses are available in a Public ⁄ Private Ventures report; see Herrera et al., 2007.) See Table 1 for baseline means for covariates and outcome mea- sures. For scales, Cronbach‘s alphas for each of the three waves (a1, a2, a3) are reported.
Table 1
Means of Baseline Covariates and Outcomes
Measure
Baseline
M (SD)
for controls
Baseline
M (SD)
for treatments
Covariates
Age 11.23 (1.66) 11.24 (1.67)
Minority (%) 60.80 64.43
Female (%) 54.36 53.98
Stress 4.41 (2.54) 4.64 (2.62)
Involvement in extracurricular
activities
2.43 (1.41) 2.40 (1.48)
Free ⁄ reduced-price lunch (%) 68.93 69.21
Teacher-reported outcomes
Classroom effort 2.77 (0.76) 2.76 (0.76)
Overall academic performance 2.47 (1.09) 2.56 (1.10)
Absence without an excuse (%) 12.22 11.63
Serious school infractions (%) 13.25 10.75
Teacher relationship quality 3.81 (0.72) 3.82 (0.71)
Youth-reported outcomes
Self-perceptions of academic
abilities
2.75 (0.64) 2.80 (0.62)
Social acceptance 2.75 (0.65) 2.81 (0.68)
Global self-worth 3.18 (0.57) 3.19 (0.54)
Parent relationship quality 3.39 (0.64) 3.38 (0.63)
Misconduct outside of school (%) 44.74 47.08
Substance use (%) 15.47 11.17
Presence of a special adult (%) 63.04 58.21
350 Herrera, Grossman, Kauh, and McMaken
School-Related Performance and Attitudes
Teachers were asked to rate youth’s overall aca- demic performance on a 5-point scale from 1 = below grade level to 5 = excellent (Pierce, Hamm, & Vandell, 1999).
Classroom Effort is a six-item subscale of the Research Assessment Package for Schools–Teachers (Institute for Research and Reform in Education, 1998) that asks teachers to rate on a 4-point scale (from 1 = never to 4 = very often) how often students demonstrate effort in the classroom on tasks, such as doing ‘‘more than is required of him ⁄ her’’ or doing ‘‘the best he ⁄ she can’’ (a1 = .90, a2 = .90, a3 = .89).
Self-Perceptions of Academic Abilities is a six-item subscale of an adapted version of the Self-Percep- tion Profile for Children (Harter, 1985) using a Lik- ert response format. The items assess youth’s estimation of their own academic competence, which research links to youth’s self-concept as well as academic achievement (Moritz Rudasill & Calla- han, 2008). Typical items include, ‘‘I do very well at my class work’’ and ‘‘I feel that I am just as smart as other kids my age.’’ Respondents were asked to rate how closely they aligned with the statements on a 4-point scale ranging from 1 = not at all true to 4 = very true (a1 = .70, a2 = .72, a3 = .73).
Problem Behaviors
Unexcused absences were measured using a sin- gle-item teacher-reported measure that indicated whether youth had been absent from school with- out an excuse in the previous 4 weeks.
Teachers also reported on school misbehavior, by answering three questions: ‘‘In the last 4 weeks in your classroom, how many times has this child: (1) been suspended; (2) been sent to the principal’s office for misbehavior; or (3) been in a fight with another child?’’(Herrera, 2004). A dichotomous var- iable was created where a value of 1 was coded if the child had engaged in any combination of the three behaviors over the past 4 weeks and a 0 was coded if there had been no incidents of such behav- iors in the past 4 weeks.
Substance Use includes four items asking about the use of alcohol, tobacco, marijuana, and other drugs. Youth reported whether they had ever used each of these substances and, if so, how frequently during the past 3 months. The items were com- bined to form a dichotomous variable where 1 indi- cates that the youth previously used any substance and 0 indicates no reported history of any sub-
stance use. The items on Substance Use are adapted from the Self-Reported Behavior Index (Brown, Cla- sen, & Eicher, 1986). The response scale and the ref- erence period (the original measure asks for a report of use in the past month) were modified for the current study.
Misconduct Outside of School is a set of five ques- tions from Brown et al. (1986; adapted by Posner & Vandell, 1994) that ask youth how often in the past 3 months they have engaged in five fairly serious out-of-school misbehaviors including, ‘‘taking something on purpose that didn’t belong to you’’ and ‘‘getting into a fight in your neighborhood.’’ Items were collapsed and transformed into a dichotomous variable where a 1 indicates that the youth had engaged in at least one instance of mis- conduct across the five behaviors in the past 3 months and a 0 indicates that the youth reported not having engaged in any of these behaviors in the prior 3 months.
Social and Personal Well-Being
Social Acceptance is a six-item subscale of the Self-Perception Profile for Children (Harter, 1985). The scale is completed by teachers and contains statements assessing how accepted youth are by their peers (e.g., ‘‘This child finds it hard to make friends,’’ ‘‘This child is popular with others his ⁄ her age’’). Respondents indicate how true they believe each statement is on a 4-point scale from 1 = not at all true to 4 = very true (a1 = .69, a2 = .75, a3 = .78). Again, we adapted the original version of the instrument to use a Likert response format.
Teacher relationship quality was measured using the short version of the Student-Teacher Relation- ship Scale, which includes 15 items from the Close With Teacher and Conflict With Teacher subscales (Pianta, 2001). This teacher-reported measure is scored on a 5-point scale, from 1 = definitely does not apply to 5 = definitely applies, with a higher score indicating a more positive relationship (a1 = .90, a2 = .91, a3 = .89). We modified one item in the Close With Teacher subscale to, ‘‘This child seems uncomfortable with personal conversations with me’’ due to agency discomfort with the original phrasing (‘‘This child is uncomfortable with physi- cal affection or touch from me’’).
Parent relationship quality was assessed using seven items from the Parent Trust subscale of the Inventory of Parent and Peer Attachment (Armsden & Greenberg, 1987). Respondents indicated the
School-Based Mentoring 351
level of support felt in the relationship with their parent ⁄ guardian. For example, youth were asked to rate how often they feel that their parent accepts them as they are or how often they feel that their parent trusts their judgment. Responses are coded on a 4-point scale, ranging from 1 = hardly ever to 4 = pretty often (a1 = .83, a2 = .87, a3 = .87).
Global Self-Worth is an eight-item subscale of the Self-Esteem Questionnaire (DuBois, Felner, Brand, Phillips, & Lease, 1996) that measures the level of youth’s self-worth. Youth respond to items such as, ‘‘I am happy with the way I can do most things’’ and ‘‘I am the kind of person I want to be’’ on a 4-point scale where 1 = not at all true and 4 = very true. Higher scores reflect more positive self-evalua- tions (a1 = .76, a2 = .80, a3 = .83).
Presence of a special adult was measure because much of the theory behind the benefits of mentor- ing postulates that the mentor becomes for the child a ‘‘special adult’’ with whom the child forms a strong attachment (DuBois, Neville, Parra, & Pugh- Lilly, 2002). However, given the shorter period of interaction and the more structured context in which the relationship develops, school-based men- tors may simply serve as providers of instrumental help and never achieve ‘‘special adult’’ status. To better understand the nature of the intervention, we assess whether involvement in the program increases the likelihood that youth felt they had a special nonparental adult in their lives. We also wanted to test whether having this type of relation- ship at baseline affected youth’s ability to benefit from involvement in the mentoring program. We asked youth the following question: ‘‘Right now in your life, is there a special adult (not your parent or guardian) who you often spend time with? A special adult is someone who does a lot of good things for you. For example someone (a) who you look up to and encourages you to do your best, (b) who really cares about what happens to you, (c) who influences what you do and the choices you make, and (d) who you can talk to about personal problems?’’
In addition to these measures, we collected data on outcome covariates, such as economic and demographic information on youths’ age, race, gen- der and free or reduced-price lunch status. Two of these variables warrant further description:
In the Stressful Life Events measure (adapted from the Social Readjustment Rating Scale by Holmes & Rahe, 1967), youth were asked if they had experi- enced, over the prior 6 months, any one of 12 events, such as ‘‘Have you moved or changed where you live?’’ and ‘‘Was someone you know well hurt badly or very ill?’’
Extracurricular involvement was measured as the sum of six items reported by youth, including after-school sports participation, after-school homework help or tutoring, and involvement in activities or clubs outside of school.
Statistical Procedures
First, we conducted statistical tests using SAS 9.1 to: (a) ensure that random assignment created two statistically equivalent groups and (b) examine the similarities between the sample using postenroll- ment data and the original randomized sample. Then, we examined the impact of SBM on the Lit- tles by comparing regression-adjusted means on all outcome variables. In particular, impacts were esti- mated (via Stata 9.2) using the following two-level random-intercept regression model, which accounts for clustering by school:
yij ¼ b0 þ b1Preij þ b2Tij þ bkXijk þ lj þ eij ð1Þ
for i = 1, . . . , nj individuals per school, j = 1, . . . , J schools, k = 1, . . . , K baseline individual level co- variates, where yij is the postenrollment outcome of interest for student i in school j, Preij is the baseline measure assessed before randomization, Tij is a dummy variable equal to 1 if student i in school j is assigned to the treatment group, Xijk is a vector of baseline student level covariates (i.e., age, minority status, receiving free or reduced-price lunch, gen- der, stress, involvement in extracurricular activities, and substance use), lj is the school-level error com- ponent, eij is the individual-level error component, and b2 is a ‘‘fixed-effect’’ estimate of the treatment effect that indicates the program effect of SBM for the average student in the sample (Mundlak, 1978).
Coefficients from these regressions are standard- ized (Hedges & Olkin, 1985) by dividing them by the standard deviation of the outcome across all students. When using multilevel models, research- ers must decide which variance should be used for standardizing the coefficients. We have used the full variance to be conservative (What Works Clear- inghouse, 2008). In cases where the outcome is measured dichotomously, we used logistic regres- sion analyses within the same random-intercept modeling framework described above. The odds ratios were standardized using the Cox index because this index appears to be the best estimator of the population standardized mean difference (Sanchez-Meca, Marin-Martinez, & Chacon-Moscoso, 2003). Differential impacts for subgroups of youth
352 Herrera, Grossman, Kauh, and McMaken
based on gender, race, age, and the presence of a special adult in the youth’s life prior to mentoring were also estimated by including interaction terms in Equation (1) between treatment status and indi- vidual level covariates.
Results
Initial Equivalence and Attrition
Preliminary t tests and chi-square analyses exam- ined the extent to which the Littles and their nonmen- tored peers were equivalent across a set of 18 baseline measures that included demographic characteristics and outcome variables. These analy- ses indicated that the two groups statistically differed on only one outcome measure at baseline: The Lit- tles were less likely than their nonmentored peers to have ever used substances prior to baseline (11% vs.
15%, respectively; v2 = 4.52, p < .05). As such, sub- stance use was included as a control in all impact analyses.
At the 9-month assessment, there was a 6% attri- tion rate among youth (5.8% among the Littles and 6.8% among their nonmentored peers), resulting in an analysis sample of 1,067 youth. Attrition analyses at the end of the first school year indicated that the ‘‘attriters’’ (i.e., those who were no longer in the sample for these analyses) differed from the ‘‘non- attriters’’ on 12 of the 18 characteristics at baseline (p < .15; see Table 2). A 15% significance level for the attrition analyses was selected to increase the power of the test to detect nonzero differences. A multivariate analysis of variance (MANOVA) on the 11 continuous baseline characteristics confirmed this result, F(10, 960) = 4.55, p < .0001. Nonrespond- ing youth were needier on several measures than those youth who remained in the sample over time. In addition, at this same point in time, teachers for
Table 2
Baseline Differences Between Attriters and Nonattriters at the 9-Month Assessment for Youth- and Teacher-Reported Data
Baseline characteristics
Youth based Teacher based
Attriter
M (SD)
Nonattriter
M (SD)
Attriter
M (SD)
Nonattriter
M (SD)
Demographics
Age 12.01 (2.26) 11.18** (1.60) 11.64 (2.17) 11.14* (1.57)
Minority (%) 70.83 62.04�� 70.73 61.74�
Female (%) 41.67 55.01* 36.59 57.11**
Risk indicators
Stress 5.01 (2.80) 4.49� (2.56) 4.90 (2.52) 4.52 (2.52)
Involvement in extracurricular activities 2.27 (1.36) 2.43 (1.45) 2.46 (1.37) 2.42 (1.43)
Free ⁄ reduced-price lunch (%) 73.72 68.76a 70.56 69.70
Outcomes
School-related performance and attitudes
Overall academic performance 2.15 (0.97) 2.54* (1.10) 2.28 (1.12) 2.55* (1.09)
Classroom effort 2.34 (0.70) 2.79** (0.76) 2.51 (0.79) 2.80** (0.75)
Self-perceptions of academic abilities 2.78 (0.60) 2.78 (0.64) 2.73 (0.60) 2.79 (0.64)
Problem behavior
Absence without an excuse (%) 19.23 11.50� 14.88 11.50
Serious school infractions (%) 20.37 11.53� 21.67 10.68**
Substance use (%) 22.22 12.72* 22.95 12.13**
Misconduct outside of school (%) 59.02 45.07 59.02 45.07**
Social and personal well-being
Social acceptance 2.54 (0.73) 2.79** (0.66) 2.65 (0.72) 2.80* (0.66)
Teacher relationship quality 3.41 (0.87) 3.84** (0.70) 3.56 (0.84) 3.85** (0.69)
Parent relationship quality 3.24 (0.73) 3.39* (0.63) 3.33 (0.66) 3.39 (0.65)
Global self-worth 3.23 (0.55) 3.19 (0.56) 3.14 (0.57) 3.19 (0.56)
Presence of a special adult (%) 66.67 60.25 58.97 60.83
Note. Youth: nattriters = 72, nnonattriters = 1,067; teachers: nattriters = 123, nnonattriters = 886. aThere was an insufficient distribution of cases in free ⁄ reduced-price lunch status among the attriters to validly calculate a chi-square value: Only 1 of 14 attriters reported receiving free ⁄ reduced-price lunch. ��p < .15. �p < .10. *p < .05. **p < .01.
School-Based Mentoring 353
886 (87.3% of the Littles and 88.3% of their nonmen- tored peers) of the 1,009 students whose teachers had completed a baseline assessment on the youth, completed the 9-month assessment. Attrition analy- ses of the teacher data yielded a pattern of results similar to those examining youth attrition: At base- line, youth of teachers who did not complete the 9-month survey were experiencing more difficulties than those whose teachers completed their surveys on 10 of the 18 characteristics, F(10, 960) = 2.59, p < .01. These results suggest that youth attriters for the 9-month assessment were needier at baseline than nonattriters.
Further analyses were conducted to examine dif- ferential attrition by the 9-month assessment between the Littles and their nonmentored peers to test whether the types of youth who attrited from these two groups differed. If this were the case, results would incorrectly suggest that any positive (or negative) change seen among participants was due to their involvement in the SBM program when the change was actually due to inherent differences between the Littles and their nonmentored peers among the individuals who continued to partici- pate in the study. Analyses comparing the baseline characteristics for the Littles and their nonmentored peers who remained in the study at the end of the school year showed no significant differences out- side the range of normal chance variation (p < .15). Differential attrition analyses for youth whose teachers did not attrit by the end of the first school year indicated that the groups differed on two char- acteristics at the p < .15 level: perceptions of aca- demic abilities (t = )1.52) and stress (t = )1.55); and on two variables at the p < .10 level: minority status (v2 = 2.74, p < .10) and substance use (v2 = 3.55, p < .10). However, results from a MA- NOVA indicated that there was not a significant pattern of overall differences between the Littles and their nonmentored peers, F(10, 844) = 1.15, p = .33. Taken together, the attrition analyses sug- gest that although the impact analyses at the 9-month assessment may omit those youth who were the neediest at baseline, they do not suffer from selection bias that would lead to falsely accepting or rejecting the null hypothesis that the program has no effect on participants.
Impacts at the 9-Month Assessment
Results of the two-level random-intercept regres- sion analyses indicate that participation in the BBBS SBM program led to improvements in two youth outcomes by the end of the first school year. Specif-
ically, Littles’ teachers reported significantly (p < .05) better overall academic performance and the Littles themselves reported more positive per- ceptions of their own academic abilities (p < .05) than their nonmentored peers. Additionally, Littles were more likely than their nonmentored peers to report having a ‘‘special adult’’ in their lives (OR = 1.34; 95% CI = 1.02, 1.76). The program’s unstandardized impacts and odds ratios, as well as their corresponding effect sizes (Hedges & Olkin, 1985) are presented in Table 3.
The impact analyses were also conducted exclud- ing the ninth-grade study participants to ensure that this unique subset of the sample did not bias the findings. Results yielded the following changes in our estimated impacts: perceptions of academic abil- ities: B = .06, p = .051; and absent without an excuse: OR = .63, 95% CI = .40, .99. In addition, Littles youn- ger than ninth grade were not significantly more likely than their nonmentored peers to report having a special adult in their lives at the end of the first school year (OR = 1.25, 95% CI = 0.95, 1.66). This likely reflects the fact that older youth in middle and high school were more likely than elementary-aged youth to be matched with adult mentors as opposed to high school student volunteers.
9-Month Impacts for Youth With Different Characteristics
Subgroup analyses were conducted to examine differential impacts at the end of the first school year by gender (boy vs. girl), race (minority vs. White), age group (elementary vs. middle ⁄ high school), and the presence of a special adult in the youth’s life. These analyses revealed no significant interactions between treatment status and gender, race or age group. Only one significant interaction between special adult and treatment group emerged (v2 = 4.95, p < .05). Littles who lacked a special adult prior to program participation were more likely than their nonmentored peers to have used substances by the 9-month assessment (OR = 2.11, 95% CI = 1.13, 3.97), while no signifi- cant treatment group difference was found for youth with a special adult at baseline (OR = .85, 95% CI = .51, 1.40).
Validity of Teacher Reports
Given that one of the significant impacts for the full sample at the end of the first school year was seen in teacher-reported academic performance, further analyses were conducted to assess the
354 Herrera, Grossman, Kauh, and McMaken
extent to which teachers may have known which students were receiving mentoring and, if they knew, whether they biased their responses in favor of the Littles. To investigate this possibility, we used three different approaches. First, in fall 2005, when we had to recontact a small number of teach- ers (31) about 97 children to clarify their classroom status from the previous school year, the teachers were asked if they knew whether the student had met with a mentor at that time. These teachers cor- rectly identified the status of either having or not having a mentor for only 38% of the children. Sec- ond, one would expect that if the mentor met with the child during the school day (as opposed to after school), the teacher would be more likely to know about it. Similarly, we hypothesized that elemen- tary school teachers would be more likely to know the status of the children than middle school teach- ers because they work with students longer each day. Thus, we might expect larger impacts for youth in school-day programs and for those in elementary school. However, two-level random- intercept regression analyses examining differential program impacts by time of program (during school vs. after school) and age (elementary vs. middle ⁄ high school) found no significant group differences across the teacher-reported outcomes.
15-Month Impacts for the Full Sample
The impact analyses at the 15-month assessment (15 months after the start of the study) revealed no
significant differences between Littles and their nonmentored peers on any of the 11 outcome mea- sures we tested, even for those that had signifi- cantly differed in the spring of the previous school year (see Table 4). In general, across the youth out- comes we assessed, Littles improved more than their nonmentored peers by the end of the first school year but then declined to levels equivalent to those of their nonmentored peers by late fall of the second school year. Littles, however, continued to be more likely than their nonmentored peers to report having a special adult in their lives (B = .37, p < .01).
15-Month Impacts for Youth With Different Characteristics
Similar to the subgroup analyses conducted at the end of the first school year, we found no strong evidence that SBM led to larger impacts for certain types of youth at the 15-month assessment. No sig- nificant interactions between treatment and gender or race emerged in the late fall of the second school year. However, one differential impact did emerge by age group: Older mentored youth (those in mid- dle and high school) demonstrated less effort in the classroom compared to their nonmentored peers (B = ).14, p < .05), and this impact was larger than that for elementary-aged Littles relative to their nonmentored peers (B = .07, p = .21), v2 = 6.93, p < .01. A significant interaction between special adult and treatment group also emerged at
Table 3
Summary of Program Impacts at the 9-Month Assessment and Relative Effect Sizes
Outcomes B SE OR 95% CI Effect size
School-related performance and attitudes
Overall academic performance (1–5) .11* 0.05 — — 0.09
Classroom effort .06 0.04 — — 0.07
Self-perceptions of academic abilities (youth) .07* 0.03 — — 0.11
Problem behavior
Absence without an excuse (0, 1)a ).42� 0.23 0.66 0.42, 1.02 –0.26
Serious school infractions (0, 1)a ).40� 0.21 0.67 0.45, 1.00 –0.24
Substance use (youth; 0, 1)a .18 0.20 1.19 0.81, 1.75 0.11
Misconduct outside of school (youth; 0, 1)a ).08 0.13 0.92 0.71, 1.20 )0.05
Social and personal well-being
Social acceptance .05 0.04 — — 0.06
Teacher relationship quality (1–5) .03 0.04 — — 0.04
Parent relationship quality (youth) .05 0.04 — — 0.07
Global self-worth (youth) .02 0.03 — — 0.03
Presence of a special adult (youth; 0, 1)a .29* 0.14 1.34 1.02, 1.76 0.18
Note. Outcomes are teacher-reported scales and range from 1 = low to 4 = high, unless otherwise noted. aEstimates reported for dichotomous outcomes are logistic coefficients. The control group is the reference group. �p < .10. *p < .05.
School-Based Mentoring 355
15 months. Littles who lacked a special adult at baseline had significantly higher perceptions of their own academic abilities than their nonmen- tored peers at the 15-month assessment (B = .12, p < .05), and this impact was significantly larger than that for youth who had a special adult at base- line, who experienced no impact in this area (B = ).02, p = .71), v2 = 3.92, p < .05.
Discussion
This large-scale random assignment impact study tested whether and in what ways the BBBS SBM program provides benefits to its youth participants by following youth for 1½ school years. It provides insight into how mentoring in the school setting may help children succeed in this context and the role that volunteer mentors might play in the lives of young people as part of their school experience. At the same time, it highlights important limita- tions of the program model as it is currently imple- mented.
In the first school year of involvement, partici- pants received about 5 months of mentoring, an amount that is fairly typical of SBM programs because they require some start-up time at the beginning of the school year and generally end prior to the end of the school year. Despite this short amount of time, relative to their nonmentored
peers, youth experienced modest academic benefits. Teachers reported small gains in academic perfor- mance, while youth reported similar improvements in perceptions of their own academic abilities. Littles, particularly those who were older and more likely to be matched with an adult, were also more likely to report having a ‘‘special adult’’ in their lives who provides them with the types of supports BBBS strives to provide through its mentors. Thus, SBM appears to be one way to increase the number of meaningful adult relationships in children’s lives, which Bergin and Bergin (2009) suggest is key in promoting healthy development.
However, academic impacts did not persist into the second school year after about half of the matches had ended. We also found no evidence at either assessment that SBM had significant effects on youth’s classroom effort, problem behaviors, or other indicators of their social and personal well- being including relationships with peers and adults and global self-esteem.
Analyses also suggest that there are very few dif- ferences in program benefits between youth across demographic subgroups (i.e., age, gender, and race) or between youth who did or did not have a pre- existing ‘‘special adult’’ in their life. And those few differences that were found were not entirely con- sistent. For example, relative to their nonmentored peers, youth without a special adult at baseline were more likely to use substances at 9 months, but also reported more positive perceptions of their own academic abilities at 15 months.
In this context, it is important to note that the younger, high-school-aged mentors in this sample were typically matched with youth in elementary school, rather than older youth. Thus, a lack of age- related findings could mean that older and younger students reap similar benefits from program involvement. However, because age of mentor and age of youth are confounded in these subgroup analyses, firm conclusions cannot be drawn without more carefully exploring the benefits yielded by high school and adult mentors separately.
The academic impacts in this evaluation support findings from a recent literature review (Portwood & Ayers, 2005) as well as several studies using non- experimental or quasi-experimental designs that have suggested SBM benefits youth in school- related areas (e.g., Diversi & Mecham, 2005; Han- sen, 2001, 2002; Herrera, 2004). Given the increasing pressure on schools and parents to do a better job at fostering children’s school success, this study suggests that the presence of mentors in schools can help students get more out of school during the
Table 4
Summary of Program Impacts at the 15-Month Assessment
Outcomes B SE
School-related performance and attitudes
Overall academic performance (1–5) .01 0.06
Classroom effort ).01 0.05
Self-perceptions of academic abilities (youth) .03 0.03
Problem behavior
Absence without an excuse (0, 1)a ).21 0.23
Serious school infractions (0, 1)a .04 0.23
Substance use (youth; 0, 1)a ).18 0.19
Misconduct outside of school (youth; 0, 1)a ).05 0.14
Social and personal well-being
Social acceptance ).03 0.04
Teacher relationship quality (1–5) ).01 0.04
Parent relationship quality (youth) .00 0.04
Global self-worth (youth) .03 0.03
Presence of a special adult (youth; 0, 1)a .36** 0.14
Note. Outcomes are teacher-reported scales and range from 1 = low to 4 = high, unless otherwise noted. aEstimates reported for dichotomous outcomes are logistic coefficients. The control group is the reference group. **p < .01.
356 Herrera, Grossman, Kauh, and McMaken
school year they are matched. The impacts we observed, although limited, were in a key area of academic success—making the intervention poten- tially valuable to schools and parents.
However, our findings do not support previous links suggested in the literature between SBM and improvements in social and personal well-being— most notably in peer relationships (Cavell & Hughes, 2000; Herrera, 2004; Karcher, 2008; King et al., 2002). It is possible that although the average youth did not improve in this domain, some youth may have experienced these benefits—for example, those youth with more peer-related needs, those in programs that allowed for more peer interactions during match meetings, or those whose mentors focused more on social outcomes. Assessing whether specific subgroups of youth experienced stronger benefits in this and other types of out- comes, and whether certain program characteristics are more conducive to these outcomes will be an important next step for the mentoring field.
Although the effect sizes in this study are mod- est, they are almost identical to those reported in Public ⁄ Private Venture’s (P ⁄ PV) 1995 impact study of BBBS CBM programs (Grossman & Tierney, 1998). However, the CBM participants benefited in several areas that did not appear to be affected in the current study (e.g., parent and peer relation- ships, substance use). Thus, 5 months of BBBS SBM led to impacts of about the same size as approxi- mately 12 months of mentoring observed in P ⁄ PV’s evaluation of BBBS CBM but in a much more lim- ited set of outcomes—specifically those related to academic performance.
Importantly however, an assessment in the mid- dle of the next school year did not find impacts on any of the outcomes tested, except that Littles, on average, continued to be more likely to report hav- ing a relationship with a supportive and caring nonparental adult. The decay in impacts likely reflects the fact that almost half of the Littles were no longer receiving mentoring in the second school year of the study. By the start of the second school year, close to one third of the Littles had transferred to a new school—typical of SBM programs that serve fifth and eighth graders (who usually transi- tion to a new school when they begin sixth and ninth grades) and of the mobility level of schools served by these and other BBBS programs.
These findings may also reflect an important methodological characteristic of the study: the tim- ing of our 15-month assessment. We chose the late fall, in part, to allow us to assess whether having a mentor in the fall could help the child ‘‘catch up’’
after the summer break and offset the summer learning loss that is experienced by most students after the break (Cooper, Nye, Charlton, Lindsay, & Greathouse, 1996).
We also believed that the first semester was a good point at which to gauge impacts because, in most schools, grades at that time point are a good indication of the student’s success over the school year. However, to assess the students prior to the end of the first semester, we needed to begin data collection in mid-November. Because most pro- grams did not start immediately when school opened, even youth who were involved in the pro- gram in the second school year of the study received only about 3 additional months of mentor- ing before the 15-month assessment. Perhaps more salient for the relationship, many continuing matches had not interacted for a 4-month period, during the summer and beginning of the school year. This relationship disruption, combined with the summer learning loss phenomenon (with both groups of youth perhaps losing academic ground over the summer months) may have impeded whatever progress the match had made in the 1st year of their meetings.
Thus, our findings suggest that the modest aca- demic benefits of 1 school year of SBM may dimin- ish soon after leaving the program. Both the high attrition from the program and the break in the mentoring relationship over the summer point to challenges in the program’s model that very likely contributed to this decay in impacts. However, our findings do not indicate whether youth who sus- tained their relationship continued to benefit from the program. The design does not enable us to address this question experimentally. This high- lights the need for future studies to rigorously test whether longer SBM matches are linked with stron- ger impacts, and if so, how many months of SBM is enough to yield impacts that are sustained past program involvement.
This study is important in that it provides reli- able, rigorously determined estimates of the effects of the BBBS SBM program. It is thus noteworthy that we were able to detect even modest academic impacts after an average of only 5 months of men- toring—that is, about 15 meetings. Prior theories of mentoring postulate that impacts occur, in large part, by inducing fundamental changes in how youth see others and themselves (e.g., Rhodes, 2002). The fact that we found these impacts after such a short amount of time, and that they were not sustained when many youth were no longer involved in the program, suggests that SBM’s
School-Based Mentoring 357
impacts may be dominated by more direct effects. For example, in some cases, school-based mentors help the child directly with school work, possibly leading to initial changes in performance. Even when the matches did not spend most of their time on tutoring or homework help (about three fourths of matches), the mentors’ presence in the school and interest in the child’s performance may have been enough to encourage more consistent school work.
Long-term, more permanent changes in youth’s school performance may rely on more fundamental changes in youth that simply do not occur for the average child in the 1st year of their SBM involve- ment. Much more research needs to be conducted to understand the mechanisms through which SBM works. This research could shed light on why these academic impacts were short-lived and how to cre- ate new programs or modify existing programs to help ensure their permanence and promote chil- dren’s long-term positive development.
It is worth noting that our analyses did not find evidence of teacher bias. These analyses were nec- essary because many teachers knew which youth were being mentored, and academic performance was reported by teachers. Analyses did not support the idea that teachers systematically inflated their assessments of youth depending on their group sta- tus or that the quality of their relationships with their students changed as a result of their group status.
Three important limitations of this study should be noted. First, although the agencies involved in this study reflect a range of sizes and structures, they were all selected in part because they are well- established and have strong relationships with the schools where the SBM programs are located. They had national guidelines for training, supervision, and support of matches and had been serving youth for several years. SBM implemented by younger, less established programs without this level of infrastructure may not yield similar impacts.
Further, participating agencies’ diversity in size, operations and geographical location suggests that findings may be extended to other BBBS agencies; however, we did not test how comparable the 10 study agencies are to the other close to 400 BBBS agencies nationwide, and we know that agencies participating in this study differ from the average BBBS agency in some dimensions. For example, they have larger budgets than the average BBBS agency, and their SBM programs are fairly well established. This study did not test the extent to
which the average BBBS agency, at a national level, would yield similar 1st-year impacts.
Second, our outcomes were assessed using strong measures of academic performance, atti- tudes and behavior, but these measures are not actual skill assessments. Nor did we rely on ‘‘blind’’ observations of students’ school behavior. Although our analyses did not reveal any evidence that teachers biased their responses to reflect bigger impacts for the Littles, these analyses were not definitive, and even a small degree of bias in teach- ers’ ratings could affect the impacts we observed. If a teacher bias did exist, it could have affected other conclusions as well. For example, if elementary teachers were more likely to have known about Little’s treatment status and biased their ratings in favor of stronger impacts, this would mask age differences favoring older Littles. Youth may have also biased their responses to report favorable out- comes because they understood that they were expected to make progress as a result of program participation. Their nonmentored peers and their teachers could have also been motivated to report less favorable adjustment to make clear the youth’s need for a mentor. The fact that teachers and youth independently reported similar impacts (i.e., aca- demic performance and youth perceptions of their own academic abilities) supports the idea that these findings reflect ‘‘true’’ changes in youth rather than bias. However, these caveats must be kept in mind when assessing the strength of these findings.
Third, we experienced attrition among study participants over the course of the study such that our final sample no longer included the neediest youth, particularly in terms of their academic per- formance. At baseline, nonattriters were perform- ing significantly better and showed greater classroom effort than those students whose partici- pation in the study ended prior to the 9-month assessment. Given these differences, it is possible that findings from this study may not generalize to more academically struggling students. Nonattrit- ers were also more likely to be female and youn- ger than attriters. However, our analyses examining differential impacts by sex and age suggest that the impacts would generalize to males and older youth.
Raising Healthy Children: Implications for Policy and Practice
There is increasing pressure on schools from par- ents and the public to do a better job at providing children with the skills they need to succeed. This
358 Herrera, Grossman, Kauh, and McMaken
study shows that while SBM (as currently operated) is not a ‘‘magic bullet’’ solution for failing schools, the presence of mentors in schools can help stu- dents get more out of school during the school year they are matched. The impacts we observed, although limited, were in a key area of school suc- cess—making the intervention potentially valuable to schools. Thus, schools and the students they serve may benefit from allowing more mentors into their buildings.
While the findings from this study identify SBM as an intervention worth exploring, our study also outlines clear limitations in the ability of the pro- gram to foster sustained impacts. We need to know more about the mechanisms through which SBM operates to speculate on how to improve the model in ways that might overcome this shortcoming. In this study, almost half of the mentors were attend- ing high school at the time of their involvement in the program. How do their impacts differ from those of adults? What kind of support is needed by these young mentors and how do programs need to structure themselves differently to ensure they benefit youth? What role does the school play in ensuring that strong matches are created? And perhaps most importantly, do the benefits of SBM outweigh its costs, particularly relative to other in- school and out-of-school time options for students? Answering these questions will help schools deter- mine how to best serve their students in this time of limited resources and increased pressure on schools to achieve. It will also help to ensure that a set of effective program practices is consistently implemented across schools and agencies to pro- vide youth with strong, long-lasting benefits that foster youth’s positive development.
References
Alexander, K. L., Entwisle, D. R., & Horsey, C. S. (1997). From first grade forward: Early foundations of high school dropout. Sociology of Education, 70, 87–107.
Armsden, G. C., & Greenberg, M. T. (1987). The Inven- tory of Parent and Peer Attachment: Individual differ- ences and their relationship to psychological well-being in adolescence. Journal of Youth and Adolescence, 16, 427– 454.
Aseltine, R. H., Dupre, M. E., & Lamlein, P. (2000). Men- toring as a drug prevention strategy: An evaluation of Across Ages. Adolescent and Family Health, 1, 11–20.
Bergin, C., & Bergin, D. (2009). Attachment in the class- room. Educational Psychology Review, 21, 141–170.
Bernstein, L., Dun Rappaport, C., Olsho, L., Hunt, D., & Levin, M. (2009). Impact evaluation of the U.S. Department
of Education’s student mentoring program: Final report (NCEE 2009-4047). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assis- tance.
Brown, B. B., Clasen, D. R., & Eicher, S. A. (1986). Percep- tions of peer pressure, peer conformity dispositions, and self-reported behavior among adolescents. Develop- mental Psychology, 22, 521–530.
Cairns, R. B., Cairns, B. D., & Neckerman, H. J. (1989). Early school dropout: Configurations and determi- nants. Child Development, 60, 1437–1452.
Cavell, T. A., & Hughes, J. N. (2000). Secondary preven- tion as context for assessing change processes in aggressive children. Journal of School Psychology, 38, 199–235.
Chall, J. S. (1983). Stages of reading development. New York: McGraw-Hill.
Cooper, H., Nye, B., Charlton, K., Lindsay, J., & Great- house, S. (1996). The effects of summer vacation on achievement test scores: A narrative and meta-analytic review. Review of Educational Research, 66, 227–268.
Diversi, M., & Mecham, C. (2005). Latino(a) students and Caucasian mentors in a rural after-school program: Towards empowering adult-youth relationships. Jour- nal of Community Psychology, 33, 31–40.
DuBois, D. L., Felner, R. D., Brand, S., Phillips, R. S., & Lease, A. M. (1996). Early adolescent self-esteem: A developmental-ecological framework and assessment strategy. Journal of Research on Adolescence, 6, 543–579.
DuBois, D. L., Holloway, B. E., Valentine, J. C., & Cooper, H. (2002). Effectiveness of mentoring programs for youth: A meta-analytic review. American Journal of Com- munity Psychology, 30, 157–197.
DuBois, D. L., Neville, H. A., Parra, G. R., & Pugh-Lilly, A. O. (2002). Testing a new model of mentoring. In G. G. Noam (Ed. in Chief) & J. E. Rhodes (Ed.), A critical view of youth mentoring (New Directions for Youth Develop- ment: Theory, Practice, and Research, No. 93, pp. 21–57). San Francisco: Jossey-Bass.
Eccles, J. S., & Barber, B. L. (1999). Student council, volun- teering, basketball, or marching band: What kind of extracurricular involvement matters? Journal of Adoles- cent Research, 14, 10–43.
Eccles, J. S., & Midgley, C. (1989). Stage ⁄ environment fit: Developmentally appropriate classrooms for young adolescents. In R. E. Ames & C. Ames (Eds.), Research on motivation in education (Vol. 3, pp. 139–186). New York: Academic Press.
Finn, J. D. (1989). Withdrawing from school. Review of Educational Research, 59, 117–142.
Fraser, B. J., & Fisher, D. L. (1982). Predicting students’ outcomes from their perceptions of classroom psycho- social environment. American Educational Research Jour- nal, 19, 498–518.
Goodenow, C. (1993). The psychological sense of school membership among adolescents: Scale development and educational correlates. Psychology in the Schools, 30, 79–90.
School-Based Mentoring 359
Grossman, J. B., & Johnson, A. (1999). Judging the effec- tiveness of mentoring programs. In J. B. Grossman (Ed.), Contemporary issues in mentoring (pp. 24–47). Philadelphia: Public ⁄ Private Ventures.
Grossman, J. B., Price, M. L., Fellerath, V., Jucovy, L. Z., Kotloff, L. J., Raley, R., et al. (2002). Multiple choices after school: Findings from the extended-service schools initiative. Philadelphia: Public ⁄ Private Ventures.
Grossman, J. B., & Rhodes, J. E. (2002). The test of time: Predictors and effects of duration in youth mentoring programs. American Journal of Community Psychology, 30, 199–219.
Grossman, J. B., & Tierney, J. P. (1998). Does mentoring work? An impact study of the Big Brothers Big Sisters program. Evaluation Review, 22, 403–426.
Hamre, B., & Pianta, R. (2001). Early teacher-child relation- ships and the trajectory of children’s school outcomes through eighth grade. Child Development, 72, 625–638.
Hansen, K. (2001). BBBSA Year 1 School-Based Mentoring grant evaluation results. Philadelphia: BBBSA.
Hansen, K. (2002). BBBSA Year 2 School-Based Mentoring grant evaluation results. Philadelphia: BBBSA.
Harter, S. (1985). Manual for the Self-Perception Profile for Children. Denver, CO: University of Denver.
Hedges, L., & Olkin, I. (1985). Statistical methods for meta- analysis. New York: Academic Press.
Herrera, C. (2004). School-based mentoring: A closer look. Philadelphia: Public ⁄ Private Ventures.
Herrera, C., Grossman, J. B., Kauh, T. J., Feldman, A. F., McMaken, J., & Jucovy, L. Z. (2007). Making a difference in schools: The Big Brothers Big Sisters School-Based Mentoring Impact Study. Philadelphia: Public ⁄ Private Ventures.
Herrera, C., Kauh, T. J., Cooney, S. M., Grossman, J. B., & McMaken, J. (2008). High school students as mentors: Findings from the Big Brothers Big Sisters School-Based Mentoring Impact Study. Philadelphia: Public ⁄ Private Ventures.
Herrera, C., Sipe, C. L., McClanahan, W. S., with Arbr- eton, A. J. A., & Pepper, S. K. (2000). Mentoring school- age children: Relationship development in community-based and school-based programs. Philadelphia: Public ⁄ Private Ventures.
Holmes, T., & Rahe, R. (1967). The Social Readjustment Rating Scale. Journal of Psychosomatic Reasoning, 11, 213– 218.
Hughes, J. N., Cavell, T. A., Meehan, B. T., Zhang, D., & Collie, C. (2005). Adverse school context moderates the outcomes of selective interventions for aggressive chil- dren. Journal of Consulting and Clinical Psychology, 73, 731–736.
Institute for Research and Reform in Education. (1998). Research Assessment Package for Schools (RAPS) manual. Philadelphia: Institute for Research and Reform in Edu- cation.
Jacobs, J. E., Lanza, S., Osgood, W. D., Eccles, J. S., & Wigfield, A. (2002). Changes in children’s self-compe- tence and values: Gender and domain differences
across grades one through twelve. Child Development, 73, 509–527.
Jimerson, S., Egeland, B., & Sroufe, L. A. (2000). A pro- spective longitudinal study of high school dropouts examining multiple predictors across development. Journal of School Psychology, 38, 525–549.
Karcher, M. J. (2005). The effects of developmental men- toring and high school mentors’ attendance on their younger mentees’ self esteem, social skills, and con- nectedness. Psychology in the Schools, 42, 65–77.
Karcher, M. J. (2008). The Study of Mentoring in the Learning Environment (SMILE): A randomized study of the effectiveness of school-based mentoring. Preven- tion Science, 9, 99–113.
Karcher, M. J., Davis, C., & Powell, B. (2002). The effects of developmental mentoring on connectedness and aca- demic achievement. The School Community Journal, 12(2), 35–50.
King, K. A., Vidourek, R. A., Davis, B., & McClellan, W. (2002). Increasing self-esteem and school connectedness through a multidimensional mentoring program. Jour- nal of School Health, 72(7), 294–299.
Laird, R. D., Jordan, K. Y., Dodge, K. A., Pettit, G. S., & Bates, J. E. (2001). Peer rejection in childhood, involve- ment with antisocial peers in early adolescence, and the development of externalizing behavior problems. Development and Psychopathology, 13, 337–354.
Lau, S., & Leung, K. (1992). Self-concept, delinquency, relations with parents and school and Chinese adoles- cents’ perception of personal control. Personality and Individual Differences, 13, 615–622.
LoSciuto, L., Rajala, A., Townsend, T. N., & Taylor, A. S. (1996). An outcome evaluation of Across Ages: An intergenerational mentoring approach to drug preven- tion. Journal of Adolescent Research, 11, 116–129.
Matzenbacher, D. L. (1999). Evaluating the effects of col- lege student mentors on children in an elementary school environment. Dissertation Abstracts International, 60(5-B), 2387.
MENTOR. (2006). Mentoring in America 2005: A snap- shot of the current state of mentoring. Retrieved April 23, 2007, from http://www.mentoring.org/downloads/ mentoring_523.pdf
Midgley, C. M., Feldlaufer, H., & Eccles, J. S. (1989). Changes in teacher efficacy and student self- and task-re- lated beliefs during the transition to junior high school. Journal of Educational Psychology, 81, 247–258.
Moritz Rudasill, K., & Callahan, C. M. (2008). Psychomet- ric characteristics of the Harter Self-Perception Profiles for Adolescents and Children for use with gifted popu- lations. Gifted Child Quarterly, 52, 70–86.
Mundlak, Y. (1978). On the pooling of time series and cross section data. Econometrica, 46, 69–86.
Parker, J. G., & Asher, S. R. (1987). Peer relations and later personal adjustment: Are low-accepted children at risk? Psychological Bulletin, 102, 357–389.
Patrick, H., Anderman, L. H., & Ryan, A. (2002). Social motivation and the classroom social environment. In
360 Herrera, Grossman, Kauh, and McMaken
C. Midgley (Ed.), Goals, goal structures, and patterns of adaptive learning (pp. 85–108). Mahwah, NJ: Erlbaum.
Pianta, R. C. (2001). Student-Teacher Relationship Scale (short version). Lutz, FL: Psychological Assessment Resources.
Pierce, K. M., Hamm, J. V., & Vandell, D. L. (1999). Expe- riences in after-school programs and children’s adjust- ment in first-grade classrooms. Child Development, 70, 756–767.
Portwood, S. G., & Ayers, P. M. (2005). Schools. In D. L. DuBois & M. J. Karcher (Eds.), Handbook of youth men- toring (pp. 336–347). Thousand Oaks, CA: Sage.
Posner, J. K., & Vandell, D. L. (1994). Low-income children’s after-school care: Are there beneficial effects of after-school programs? Child Development, 65, 440– 456.
Rhodes, J. E. (2002). Stand by me: The risks and rewards of mentoring today’s youth. Cambridge, MA: Harvard Uni- versity Press.
Rhodes, J. E., Contreras, J. M., & Mangelsdorf, S. C. (1994). Natural mentor relationships among Latina ado- lescent mothers: Psychological adjustment, moderating processes, and the role of early parental acceptance. American Journal of Community Psychology, 22, 211–228.
Rhodes, J. E., Grossman, J., & Resch, N. (2000). Agents of change: Pathways through which mentoring relation- ships influence adolescents’ academic adjustment. Child Development, 71, 1662–1671.
Sanchez-Meca, J., Marin-Martinez, F., & Chacon-Moscoso, S. (2003). Effect-size indices for dichotomous outcomes in meta-analysis. Psychological Methods, 8, 448–467.
Seidman, E., Allen, L., Aber, J. L., Mitchell, C., & Fein- man, J. (1994). The impact of school transitions in early adolescence on the self-system and social context of poor urban youth. Child Development, 65, 507–522.
Slicker, E. K., & Palmer, D. J. (1993). Mentoring at-risk high school students: Evaluation of a school-based pro- gram. The School Counselor, 40, 327–334.
Spreen, O. (1978). Prediction of school achievement from kindergarten to grade five: Review and report of a follow-up study (Research Monograph, No. 33). Victo- ria, Canada: Department of Psychology, University of Victoria.
Spreen, O. (1988). Learning disabled children growing up: A follow-up into adulthood. New York: Oxford University Press.
Timmer, S. G., Eccles, J., & O’Brien, I. (1985). How children use time. In F. T. Juster & F. P. Stafford (Eds.), Time, goods and well-being (pp. 353–382). Ann Arbor: University of Michigan, Institute for Social Research.
What Works Clearinghouse. (2008, December). Procedures and standards handbook (version 2.0). U.S. Department of Education, Institute of Education Sciences. Retrieved October 25, 2009, from http://ies.ed.gov/ncee/wwc/ help/idocviewer/Doc.aspx?docId=19&tocId=8#hlm
School-Based Mentoring 361
This document is a scanned copy of a printed document. No warranty is given about the accuracy of the copy.
Users should refer to the original published version of the material.
